Developing a Taste for Interactions
KK Moore, Senior Master Black Belt, Shaw Industries Group, Franklin, TN, USA
Keywords: Design of Experiments, DOE, Training/Education
When I teach experimental design concepts, I could discuss varying machine settings such as machine speed, temperature, pressure, product mixes, etc. to see how each one affects some important output measure such as production rates or off-quality. Or, I could show the students how these concepts come alive by varying the basic recipe of my cinnamon rolls. For five years in a row, I’ve been literally helping students develop a taste for interactions. I really enjoy what has now become an annual tradition of demonstrating this Lean Six Sigma tool.
Each year, I come up with a different theme. Given the fact that the next Lean and Six Sigma World Conference is being held in Tennessee, perhaps it would be most appropriate to share the results of the cinnamon roll DOE that tested different orange variations.
In this session, participants will learn how to:
1. Apply experimental design concepts in the classroom to increase student participation and understanding.
2. Analyze and interpret a fractional factorial screening design.
3. Develop a taste for interactions by exploring confounding schemes of a resolution III design
Here are the questions I wanted this experiment to answer:
1. Is an orange-flavored dough preferred to the basic plain dough that I normally use?
2. Is an orange-cream cheese icing better than the plain sugar glaze that I normally use?
3. Is it OK to freeze the cinnamon rolls? Here, I was speculating that freezing the cinnamon rolls wouldn't necessarily make them taste better. However, if the test subjects couldn't tell a difference, that would be a "win" for me. I would be able to make these rolls well in advance for my friends and co-workers instead of baking at 5:00 am before heading to work.
4. Is an individual serving of a cinnamon roll that is baked in a muffin cup preferred to one that is taken from a batch baked in a larger pan that I normally use?
5. Do UT fans have different preferences re: cinnamon rolls than those who are not UT supporters? Here, I was speculating that orange-blooded folks would prefer the orange variations . . . no matter what. In contrast, I was speculating that the non-UT fans would say they didn't like any of the orange variations... no matter what.
These questions were translated to the following DOE factors and levels:
Factor: Current Condition (-): Change to Try (+):
A: Dough Plain Orange
B: Icing Plain sugar glaze Orange cream cheese icing
C: Frozen Not frozen Frozen
D: Muffin Cup No muffin cup Muffin cup
E: UT Fan Not a UT UT fan
The five factors were tested using a 2^5-2 fractional factorial design with 8 unique treatment combinations.
The first four batches were made fresh on the morning of class:
Batch 1: Plain dough with plain sugar glaze icing that has been baked in a muffin cup, tasted by a UT fan
Batch 2: Orange dough with plain sugar glaze icing that has not been baked in a muffin cup, tasted by a non-UT fan
Batch 3: Plain dough with orange cream cheese icing that has not been baked in a muffin cup, tasted by a UT fan
Batch 4: Orange dough with orange cream cheese icing that has been baked in a muffin cup, tasted by a non-UT fan
The last four batches were made a few weeks prior to class and frozen. I took them out of the freezer to thaw the night before class:
Batch 5: Plain dough with plain sugar glaze icing that has been baked in a muffin cup, tasted by a non-UT fan
Batch 6: Orange dough with plain sugar glaze icing that has not been baked in a muffin cup, tasted by a UT fan
Batch 7: Plain dough with orange cream cheese icing that has not been baked in a muffin cup, tasted by a non-UT fan
Batch 8: Orange dough with orange cream cheese icing that has been baked in a muffin cup, tasted by a UT fan
Each student/co-worker could sample as few as one or as many as four different batches. Then, each completed a survey for each batch tasted, providing a “likeability” score (using a 1-5 scale). Eight surveys were collected from each batch, for a total of 64 observations used in the analysis.
After analyzing the results, I learned that baking in a muffin cup and freezing the cinnamon rolls significantly affected the likeability score. Even though most of the responses were very favorable, I learned that using a muffin cup (Factor D) had a negative effect on the likeability score. That is, the average likeability score from those batches where I used the muffin cup was statistically significantly lower than the average likeability score from those batches that I baked in a regular large pan. Hindsight, I think this makes sense. I know that I prefer the gooey centers as opposed to the outer crusts. The rolls baked in muffin cups had less gooey centers and more outer crusts. In addition to the muffin cups, I learned that freezing the cinnamon rolls (Factor C) also had a negative effect on the likeability score.
Interestingly, it appears that the average likeability score is about the same, whether or not you use orange cream cheese icing or the plain sugar glaze icing (Factor B). Also, there was essentially no difference in the likeability score among the non-UT fans and the UT fans (Factor E). That surprised me!
Although changing the dough (Factor A) wasn't technically significant, the average likeability score from those batches with the orange dough appears to be heading in the right direction. I kept exploring. Then I uncovered that changing to the orange dough actually produced more consistent results. Scores from the plain dough ranged anywhere from 1 to 5 (all over the place!); whereas, scores from the orange dough only ranged from about 3 to 5.
For this experiment, I found a clue where possibly the dough and the icing interact. That is, what happens when you combine these two variables? Developing a taste for interactions (pun intended) is one of the most important takeaways from this simulation. The data suggests that if I use the plain sugar glaze icing, then it would be best if I used the orange dough. If, however, I use the orange cream cheese icing, then it doesn't matter too much what type of dough I use. Even more surprising is the combination of the plain sugar glaze icing with the plain dough (the status quo condition). This produced the lowest likeability score!
Bottom line: I definitely want to avoid freezing the rolls and avoid using muffin cups. I definitely will continue to share my rolls with the diverse population of friends and co-workers (orange-blooded or not). At this point, I will stick with the basic sugar glaze icing. Since icing didn't matter much, why incur the added expense of cream cheese? It's the dough that I'm still a little uncertain about. So, that means more testing. I'm guessing that my students and co-workers will have no problem with that!