The End of Theory: The Data Deluge Makes the Scientific Method Obsolete by Chris Anderson

I must say I am not at all convinced that a new method without theory ready to supplant the existing scientific method. Now I can’t find peter Norvig’s exact words online (come on Google - organize all the world’s information for me please). If he said that using massive stores of data to make discoveries in new ways radically changing how we can learn and create useful systems, that I believe. I do enjoy the idea of trying radical new ways of viewing what is possible.

Practice Makes Perfect: How Billions of Examples Lead to Better Models (summary of his talk on the conference web site):

Related: Will the Data Deluge Makes the Scientific Method Obsolete? - Pragmatism and Management Knowledge - Data Based Decision Making at Google - Seeing Patterns Where None Exists - Manage what you can’t measure - Data Based Blathering - Understanding Data - Webcast on Google Innovation
(more…)

I have been tagged by Mark Graban of the lean blog: “Tag” - 5 Things You Don’t Know About Me.

• I spent a year in Singapore and another in Nigeria while I was growing up.
• Dad, Bill Hunter, was a professor (related to the item above), who co-authored Statistics for Experimenters and applied Deming’s ideas in the Public Sector for the first time. Out of the Crisis pages 245-247 include a write up on that effort with the First Street Garage. Peter Scholtes, at the time worked for the City of Madison, and played a big part in the effort. He went on to write the Team Handbook and The Leader’s Handbook.
• I was on the Wisconsin Badger Basketball camp championship teams in 7th and 8th grade. The second year we played the championship game on the regular Badger Basketball home court. The Badger’s are a bit better now then they were then.
• I have flown on “Air Force One.” Not technically, since it the president was not aboard, but while working for the White House Military Office I flew on the plane on a couple test flights. It is officially “Air Force One” only when the President is flying.
• I spent many Thanksgivings beating John Dower, my father (and other of the family members of both) at Oh Hell. Some might claim I remember more victories today than took place at the time.

John Hunter. The small person is me, the bigger one is Dad.

I tag: Kathleen Fasanella, Mike Wroblewski, Peter Abilla, Karen Wilhelm and John Dowd.

More on Madison’s Quality efforts: Doing More With Less in the Public Sector: A Progress Report from Madison, Wisconsin - Quality in the Community: One City’s Experience

Dr. George Box is not as well known in the general management community as his ideas merit (in my biased opinion - photo of Bill Hunter and George Box). He is well know in the statistics field as one of the leading statistical minds. Box on Quality is an excellent book that gathers his essays from his 65th to 80th year. The book has just been issued in paperback (which helps as the hardback was pricey).

While some of the essays are aimed at a reader with an advanced understanding of statistics, many of the articles are aimed at any manager attempting to apply Quality Management principles (SPC, Deming, process improvement, six sigma, etc.). An except from the book provides a table of contents and an introduction.

Some of the articles from the book are available online. I encourage you to take a look at several of the articles and then go ahead and add this book to your prized management resources, if you find them worthwhile.

• Good Quality Costs Less? How Come?
  
• When Murphy Speaks -Listen
  
• How to Get Lucky
• Six Sigma, Process Drift, Capability Indices, and Feedback Adjustment (with Alberto Luceneo)
• Statistics as a Catalyst to Learning
• Teaching Engineers Experimental Design With a Paper Helicopter

Comments on Who Influences Your Thinking? - Survey results -

> 1. Are people getting most of their information
> from other sources?
That would be my guess.

Similar to the phenomenon of “the long tail” which is an interesting topic in its own right. We tend to focus on the popular few (books, musicians, movies, authors, computer programs…) but often the sum of the less popular many is more significant. See:

• The Long Tail, by Chris Anderson, Wired, Oct 2004 “The average Barnes & Noble carries 130,000 titles. Yet more than half of Amazon’s book sales come from outside its top 130,000 titles. Consider the implication: If the Amazon statistics are any guide, the market for books that are not even sold in the average bookstore is larger than the market for those that are”
• Continued discussion of the Amazon figures in a Chiris Anderson’s blog. “I’ve now spoken to Jeff Bezos (and others) about this. He doesn’t have a hard figure for the percentage of sales of products not available offline, but reckons that it’s closer to 25-30%.”
• The long tail - a secret sauce for companies like Amazon.com, Netflix and Apple Computer, Motley Fool, NPR Audio Recording

Getting back to the question raised by the “Who Influences Your Thinking” post; More importantly I believe they (we) are just failing to get all we should.
(more…)

The classic Statistics for Experimenters has been updated by George Box and Stu Hunter, two of the three original authors. Bill Hunter, who was my father, and the other author, died in 1986. Order online: Statistics for Experimenters: Design, Innovation, and Discovery , 2nd Edition by George E. P. Box, J. Stuart Hunter, William G. Hunter.
I happen to agree with those who call this book a classic, however, I am obviously biased.

Google Scholar citations for the first edition of Statistics for Experimenters.
Citations in Cite Seer to the first edition.

The first edition includes the text of Experiment by Cole Porter. In 1978 finding a recording of this song was next to impossible. Now Experiment can be heard on the De-Lovely soundtrack.

Text from the publisher on the 2nd Edition:
Rewritten and updated, this new edition of Statistics for Experimenters adopts the same approaches as the landmark First Edition by teaching with examples, readily understood graphics, and the appropriate use of computers. Catalyzing innovation, problem solving, and discovery, the Second Edition provides experimenters with the scientific and statistical tools needed to maximize the knowledge gained from research data, illustrating how these tools may best be utilized during all stages of the investigative process. The authors’ practical approach starts with a problem that needs to be solved and then examines the appropriate statistical methods of design and analysis.
(more…)

Originally posted to the Deming Electronic Network, 22 Sep 1999, in reponse to this message

I would like to say that I think it is good that we have disagreements on the DEN. I think it is a strength of the DEN, not a weakness. However, I think we sometimes get to personal with no real purpose. One example of this, for me, is: “Well, I guess we knew different Demings. Mine was a teacher named Dr. W. Edwards Deming.” I doubt this statement is meant to be taken literally, and if it is not I do not see what it adds to the discussion. I point this out not because I think this is some bad act that should be punished but that I think we need to continue to develop a sense of how we wish to express our disagreements and I think that we should try to do so more constructively.

I find this statement so far from the truth that it would seriously damage any PDSA with this as an accepted assessment of history. I do not believe Deming had such an inaccurate view (of course I may be wrong). I do believe we need to improve our practice of Quality (and to do that we need to understand what happened in the past and why it was not more successful). The idea that Design of Experiments (DoE) was at the core of some Quality Movement to me is not at all accurate. (more…)