Category Archives: Design of Experiments

Scientific Thinking – the Modern Way

“Scientific thinking” the modern way [the broken link was removed] by Bill Harris:

What does this all mean? It simply means that Fisher’s designed experiments give us better and faster means to extract insight from tests on system dynamics models than the old one-factor-at-a-time approach.

I thank Deb Schenk, then (and perhaps now) statistician at Hewlett-Packard Company, for teaching me and others about the design of experiments using Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building back in 1981-82.

I admit to a bit of bias, in seeing my father’s book (Statistics for Experimenters 2nd edition was published last year by the way), referenced but Bill Harris is exactly right in the power of design of experiments. The most recent post [the broken link was removed] discusses Ackoff’s excellent f-Laws and a previous post discusses Deming (titled, It’s the process [the broken link was removed]) so I couldn’t resist adding a post myself.

Related: design of experiments postsAckoff’s New Book: Management f-Laws

Why Use Designed Factorial Experiments?

One-Factor-at-a-Time Versus Designed Experiments (site broke link so I removed it -when will people learn how to manage web content?) by Veronica Czitrom:

The advantages of designed experiments over [One Factor at a Time] OFAT experiments are illustrated using three real engineering OFAT experiments, and showing how in each case a designed experiment would have been better. This topic is important because many scientists and engineers continue to perform OFAT experiments.

I still remember, as a child, asking what my father was going to be teaching the company he was going to consult with for a few days. He said he was going to teach them about using designed factorial experiments. I said, but you explained that to me and I am just a kid, how can you be teaching adults that? Didn’t they learn it in school? The article is a good introduction to the idea of why one factor at a time experiments are an ineffective way to learn.

Related: Design of Experiments articlesStatistics for Experimenters (2nd Edition)Design of Experiments blog posts

Using Design of Experiments as a Process Road Map

Using Design of Experiments as a Process Road Map by Davis Balestracci:

The current design of experiments (DOE) renaissance seems to favor factorial designs and/or orthogonal arrays as a panacea. In my 25 years as a statistician, my clients have always found much more value in obtaining a process “road map” by generating the inherent response surface in a situation. It’s hardly an advanced technique, but it leads to much more effective optimization and process control.

DOE is a tool that is very useful. And while the situations in which DOE is the best tool to use is limited the limited use of DOE is used less than it could be. See more articles on the use of design of Experiments (DOE).

Using Design of Experiments

How to Institute DOE in Your Company (link broken – removed) by Davis Balestracci:

DOE works, but I don’t need to sell that to the readers of this newsletter. But as certain as we all are, no one can deny that design of experiments faces resistance even in environments where it is a proven tool. Every research scientist or engineer who has had a major success from DOE can tell you story after story of how management still wanted problems solved one-factor-at-a-time.

Design of Experiments (DoE) was developed by R.A. Fisher in the 1920s (related terms: factorial design, multivariate expertness). Six Sigma was the first general management approach that specifically highlighted the use of Designed Experiments for improvement. Still the use of factorial designed experiments is much less than it could be.
Continue reading

Box on Quality

Bill Hunter and George Box

Dr. George Box is not as well known in the general management community as his ideas merit (in my biased opinion – photo of Bill Hunter and George Box). He is well know in the statistics field as one of the leading statistical minds. Box on Quality is an excellent book that gathers his essays from his 65th to 80th year. The book has just been issued in paperback (which helps as the hardback was pricey).

While some of the essays are aimed at a reader with an advanced understanding of statistics, many of the articles are aimed at any manager attempting to apply Quality Management principles (SPC, Deming, process improvement, six sigma, etc.). An except from the book provides a table of contents and an introduction.

Some of the articles from the book are available online. I encourage you to take a look at several of the articles and then go ahead and add this book to your prized management resources, if you find them worthwhile.

Design of Experiments in Advertising

How Two Guys From the Gold Country Are Changing Advertising Forever [the broken link has been removed] by Robert X. Cringely

James Kowalick and Mario Fantoni, two guys who say they can show you how to use science to design ads that cost less while being 10 or more times as effective as doing it the old way.

Their secret is the Taguchi Method, which is a technique for designing experiments that converge on an ideal product solution.

“I taught over 300 courses for industry where we designed cars and electronic devices, but it wasn’t until one day I took over my wife’s kitchen and used Taguchi to perfect my recipe for vanilla wafer cookies that I realized how broadly it could be applied,” Kowalick recalls. “It took 16 batches, but by the end of the afternoon I had those wafers dialed in.”

It is great to see the application of Designed Experiments increasing. I am reminded of an article by my father, William G. Hunter, from 1975: 101 Ways to Design an Experiment, or Some Ideas About Teaching Design of Experiments. Examples of the topics of the designed experiments his students performed:
Continue reading

Design of Experiments Articles

We have added several Design of Experiments articles to the Curious Cat Management Improvement Library recently, including:

See more Design of Experiments related online resources.

Marketers Are Embracing Statistical Design of Experiments

Marketers Are Embracing Statistical Design of Experiments (site broke link so I removed it) by Richard Burnham.

Crayola® conducts an e-mail marketing DOE to attract parents and teachers to their new Internet site. The company discovers a combination of factors that makes their new e-mail pitch three-and-a-half times more effective than the control. (Harvard Business Review, October 2001, “Boost Your Marketing ROI with Experimental Design,” Almquist, Wyner.)

Marketers can’t always be certain what triggers buyers to respond. In the past, we were always admonished to test-test-test, but only one factor at a time – relying on our gut feelings and uncertain hopes. With DOE, marketers have replaced voodoo with the science of statistics.

For more on Design of Experiments see:

Open Source Management Terms

Statistical Method Helps Boost Bottom Lines, Batting Averages by Jon Van, Chicago Tribune:

Using statistical techniques embraced by quality guru W. Edwards Deming, Holland has worked for a generation guiding enterprises large and small to boost efficiency.

Yet getting management support for an MVT project is vital, Holland said, because experience shows that only 25 percent of ideas intended to improve a process will have a positive effect. The others will either have no effect or will hurt. Managers hate to see experimental results shoot down ideas they were certain would help, he said.

The article also mentions:

While Holland claims MVT as his own, others say it is really just a variation of strategies widely used in business.

“Multifactor experiments have been around for a long time,” said Ajit Tamhane, Northwestern University professor of statistics and industrial engineering and management sciences.

David Coit, a Rutgers University professor of industrial systems engineering, said that Holland’s MVT is very much like a quality-enhancing scheme called design of experiments.

So often we seem to focus on proprietary solutions. Instead it seems to me, most often what is needed is to do a good job of applying the ideas that have been known for decades. Deming ideas, design of experiments, lean thinking, experimentation, etc. are not secrets. There is a long history of how to apply these ideas to improve organizational performance.

QualPro obviously does well marketing itself (see press clippings from their web site) selling the concept of proprietary solutions to press organizations. Raising the question of whether the proprietary solutions really offers unique ideas, as the The Tribune article did was uncommon in my experience.

I like those encouraging the adoption of statistical tools to improvement management but I find the practice of trademarking terms like Six Sigma and MVT a bad way to encourage innovation in the practice of management. While it is nice to see Six Sigma efforts and others use statistical tools (such as design of experiments) I would encourage people to stay with “open source” management terms and remain part of a community looking to improve the practice of management.

Update: Also see – Management Advice Failures

Statistics for Experimenters – Second Edition

Buy Statistics for Experimenters

The classic Statistics for Experimenters has been updated by George Box and Stu Hunter, two of the three original authors. Bill Hunter, who was my father, and the other author, died in 1986. Order online: Statistics for Experimenters: Design, Innovation, and Discovery , 2nd Edition by George E. P. Box, J. Stuart Hunter, William G. Hunter.
I happen to agree with those who call this book a classic, however, I am obviously biased.

Google Scholar citations for the first edition of Statistics for Experimenters.
Citations in Cite Seer to the first edition.

The first edition includes the text of Experiment by Cole Porter. In 1978 finding a recording of this song was next to impossible. Now Experiment can be heard on the De-Lovely soundtrack.

Text from the publisher on the 2nd Edition:
Rewritten and updated, this new edition of Statistics for Experimenters adopts the same approaches as the landmark First Edition by teaching with examples, readily understood graphics, and the appropriate use of computers. Catalyzing innovation, problem solving, and discovery, the Second Edition provides experimenters with the scientific and statistical tools needed to maximize the knowledge gained from research data, illustrating how these tools may best be utilized during all stages of the investigative process. The authors’ practical approach starts with a problem that needs to be solved and then examines the appropriate statistical methods of design and analysis.
Continue reading