So I’ve mentioned before that I like pound cake, and that I’m working on a recipe. To make this go faster, for a few months I made pound cakes at home, cut them in half, and served half to my coworkers. Based on feedback from me, my wife, my friends, and coworkers (too dry / too moist / just right; more vanilla / less vanilla / you use vanilla?), and based on events during the cooking (collapses, crust cracks, etc.) I made changes to the recipe, which I tried the next time.
At first I thought of this as the scientific method, or at least engineering. And in one sense it is: I’m doing the minimum required to perform science, which is identifying a subject matter, establishing a procedure to study it, taking careful notes about the study procedure, and recording the results. And I’m doing the minimum required to perform engineering, which is identifying the is
But later I realized that my procedure is more like alchemy: tweaking something again and again without a true theory in the hope that tweaking it will somehow make gold.
To truly make it “scientific”, I’d need much more. At a minimum, I’d need to make my independent and dependent variables explicit. The independent variables are the things that I control, like the recipe, whereas the dependent variables are the outcomes, like whether the cake collapses and how it tastes. To determine the true sources of power, I’d need to change just one independent variable at a time, such as the number of eggs. To control for confounding factors in ingredients, I’d need to make two cakes at the same time, one “control” cake with the old recipe and one “experimental” with the new recipe. Furthermore, the evaluation should be double blind: I should give slices of the cakes to someone without either me or them knowing at the time which recipe they got, so taste and texture would be evaluated without the knowledge of how the results “should” turn out. Each recipe comparison should be done multiple times to control for the small-n factor. And going beyond this, other things ought to be varied, like temperature, cooking time, egg and flour varieties, etc…
Ultimately, the goal of many such experiments would be a working theory of pound cake baking: what pound cakes are, how they are baked, and what role each ingredient and each baking step has in producing the cake. Only with such a working theory could you actually do true engineering. Engineering is not science; its goal is not understanding. Instead, the goal of engineering is to take a problem description – produce a good pound cake that satisfies my late-night sweet tooth – and use the best available understanding to produce the best possible solution to the problem. Unlike a scientist digging into the unknown, an engineer’s task is to think through all the implications of the known for any potential solution to the problem. With a working theory of pound cake baking, an engineer can tell me how large a cake I can bake, whether it is feasible to bake a cake in the ovens that are likely to be available to me, and perhaps even the optimium size of pound cake for the heating characteristics of my oven. If this was a real engineering problem, a well-trained engineer would automatically go further, inquiring about the rate of pound cake consumption and the expected shelf life of baked cake, and might end up suggesting that I bake a smaller (or larger) cake so that I get the most pound cake for the least baking effort, while still ensuring that I eat it all before it goes stale.
Obviously, I’m not going to do all that. I’m going to tweak my recipe until it works right, then bake that cake and eat it. But if somehow I get tossed over into the Groundhog Day universe, I’ve got a plan to make the Best. Cake. Ever. All it will take is ten thousand trials…