|
|
|
|
|
|
Recommended posts: Mistakes in Experimental Design and Interpretation - Measurement and Data Collection - Quality, SPC and Your Career - Performance Measures and Statistics Course Materials
An Essential Primer on Full and Fractional Factorial Test Design
Even if you are using full factorial to get the same amount of information as a fractional factorial test, it will take more time since you need more data to see statistically relevant differences between the many experiments. You might be wondering how fractional factorial can be accurate if interactions are possible?
Random interactions of high relevance are very rare, especially when looking for interactions of more than 2 factors. You really need to design tests where you look for meaningful interactions that are based on true business requirements rather than hoping for a random and low influence interaction between a red button, a hero shot and a headline.
I am a fan of design of experiments as long time readers know (see posts on design of experiments).
Some good resources for more on the topics discussed above: What Can You Find Out From 8 and 16 Experimental Runs? by George Box - Statistics for Experimenters - Design of Experiments in Advertising.
Related: Google Website Optimizer - factorial experiment articles - Using Design of Experiments - Marketers Are Embracing Statistical Design of Experiments
The Rationale of Scientific Experimentation by John Dowd explains the value of designed experiments.
In addition to their efficiency, factorial designs also offer the only method of detecting interactions through experimentation. Because numerous factors can be combined in the same series of experimental runs, the interactions can be detected and the nature of their effects can be evaluated when they are present.
The paper also explains analytic and enumerative studies. Dr. Deming stressed the importance of understanding the distinction between the two.
Related: management improvement articles - Design of Experiments articles - Statistics for Experimenters - search statistical management improvement sites - Using Design of Experiments
Another interesting experiment from Google: Using Prediction Markets to Track Information Flows: Evidence from Google
Interesting paper. I would guess most readers of this blog won’t be able to apply prediction markets to there workplace in the short term but never-the-less I find the paper interesting.
Related: Management is Prediction - Google Experiments Quickly and Often - Secrets of the World’s Best Companies
I am reading a fascinating book by Jessica Snyder Sachs: Good Germs, Bad Germs. From page 108:
This is a great example of a positive special cause. How would you identify this? First you would have to stratify the data. It also shows that sometimes looking at the who is important (the problem is just that we far too often look at who instead of the system so at times some get the idea that it is not ok to stratify data based on who - it is just be careful because we often do that when it is not the right approach and we can get fooled by random variation into thinking there is a cause - see the red bead experiment for an example); that it is possible to stratify the data by person to good effect.
The following 20 pages in the book are littered with very interesting details many of which tie to thinking systemically and the perils of optimizing part of the system (both when considering the system to be one person and also when viewing it as society).
I have recently taken to reading more and more about viruses, bacteria, cells, microbiology etc.: it is fascinating stuff.
Related: Science Books by topic - Data Can’t Lie - Understanding Data
This is a pretty counter-intuitive statement, I believe:
But some simple math shows it is true. If you drive 10,000 miles you would use: 667 gallons, 556 gallons, 200 gallons and 100 gallons. Amazing. I must admit, when I first read the quote I thought that it must be an wrong. But there is the math. You save 111 gallons improving from 15 mpg to 18 mpg and just 100 improving from 50 to 100 mpg. Other than those of you who automatically guess that whatever seems wrong must be the answer when you see a title like this I can’t believe anyone thinks 15 to 18 mpg is the change that has the bigger impact. It is great how a little understanding of math can help you see the errors in your initial beliefs. Via: 18 Is Enough.
It also illustrates that the way the data is presented makes a difference. You can also view 100 mpg as 1/100 gallon per mile, 2/100 gallons per mile, 5.6/100 gpm and 6.7 gpm. That way most everyone sees that the 6.7 to 5.6 gpm saves more fuel than 2 to 1 gpm does. Mathematics and scientific thinking are great - if you are willing to think you can learn to better understand the world we live in every day.
Related: Statistics Don’t Lie, But People Can be Fooled - Understanding Data - Seeing Patterns Where None Exists - Optical Illusions and Other Illusions - 1=2: A Proof
This is a nice article discussing how people are often fooled by thinking there must be special causes for patterns in random data. I still remember my father showing my classes these lessons when I was in grade school. Playing At Dice - What That “Weekend Exercise” Was All About:
When people are asked to explain random variations in data they will make up special causes (that they often even believe are special causes even when they are not) but you can improve management a great deal by just stopping the requirement to “explain” common cause variation (which in practices mean to claim a special cause for the common cause variation). Use that time instead to standardize processes. Create control charts for critical processes. Run experiments using PDSA cycle…
Related: Seeing Patterns Where None Exists - Understanding Data - Operational Definitions and Data Collection - Red Bead Experiment
T.N. Goh received ASQ Statistics Division’s 2007 William G. Hunter Award. He sent me this email:
Also, statistical design of experiments has assumed an increasingly important role in performance improvement and optimization in the face of constrained resources, again something always in the minds of engineers, managers and business leaders. From time to time there are others who package statistical tools under labels Bill might not even have seen himself, such as “Design for Six Sigma“, but the underlying idea is still the same: recognize the existence of variation, and the earlier you anticipate it and do something about it, the better off you will be in the end.
Bill’s zeal in spreading the message and sharing his knowledge and expertise with people in other parts of the world is well known; I would even say that he had recognized that “the world is flat” way before the likes of Tom Friedman discovered the reality of globalization!
So that’s to share my thoughts with you, having being honored by the Bill Hunter award. I am copying this to Stu, also to Doug who chairs the committee for this award. I reality enjoy the professional association and friendship with you all.
I had not realized Dad was helping set up the first school of engineering in Singapore. This is the kind of thing I mentioned in, The Importance of Management Improvement, where I mention people telling me the positive impact Dad had on their lives.
Related: Curious Cat Science and Engineering Blog - Statistics for Experimenters - Singapore Research Fellowship
The Seven Fatal Flaws of Performance Measurement by Joseph F. Castellano, Saul Young, and Harper A. Roehm
Once the targets are established, most organizations measure the performance of component units by comparing targets to actual performance for certain time periods. Variances from expected results are noted and explanations are required. The popular business press trumpets the efficacy of the above approach, but this methodology has serious flaws. In fact, the design and use of performance measurement systems in most organizations suffer from a number of fatal flaws that can undermine an organization’s ability to use its measurement system to improve processes and make better decisions.
…
The proper role of measurements should be seen in the context of helping employees connect with the overall aim of the organization. Management must gather and analyze information that will help employees become better contributors to the firm’s purpose.
The articles does an excellent job of explaining the flaws in how performance measurement is applied (both in Management by Objective (MBO) and performance appraisals).
Related: Performance Without Appraisal - Jeffrey Pfeffer on Evidence-Based Practices - Problems Caused by Performance Appraisal - The Danger of Forgetting the Proxy Nature of Data
Last month, in a long post criticizing the ACSI I took issue with, among other things, the implications being drawn from an ACSI rating. The ACSI rating of Yahoo was higher than that of Google (though statistically insignificantly so). Anyway, here is some new data on search volumes of the leading providers:
Top 5 Search Providers for August 2007, Ranked by Searches (U.S.)
| Provider | Searches (000) | Year over Year Growth | Share of Searches |
|---|---|---|---|
| 1. Google Search | 4,199,495 | 39.8% | 53.6% |
| 2. Yahoo! Search | 1,561,903 | 8.9% | 19.9% |
| 3. MSN/Windows Live Search | 1,011,398 | 69.8% | 12.9% |
| 4. AOL Search | 435,088 | 32.4% | 5.6% |
| 5. Ask.com Search | 136,853 | 0.0% | 1.7% |
Source: Nielsen//NetRatings MegaView Search
So Google grew 39.8% year over year and Yahoo grew 8.9% year over year. Google now has 53.6% of the total searches. Granted this is limited data but it seems to confirm that Google is in fact continuing to increase their lead in search volume. Practically all evidence seems to support this belief - the ACSI seems to be the exception. Which might indicate great insight provided by the ACSI that everyone else is missing. Or it might show ACSI results are doing a poor job of providing a useful measure of customer satisfaction with search engines. I go with the second.
Related: posts on using data effectively - Website Data - Understanding Data - posts on Google management - Curious Cat Management Improvement Search
If organizations just adopt management improvement practices I firmly believe customer service, financial performance and employee satisfaction could be improved. This was a big part of the reason I started to use the internet to share management improvement ideas back in 1996 (plus I find management improvement interesting).
On the note of making a difference in people’s lives. I have had far more people tell me how my father (Bill Hunter) made a huge difference in their lives (far more than ever tell me anything like that). Now there is the sensible explanation, that he actually had a big impact on people’s lives (but you also have to figure most of those people never saw me so the chance for them to say anything didn’t exist…). I believe far more people told me (after he died) than ever told him, which says something about psychology in the USA, I think. But I don’t really know what people told him - so I could be wrong about that.
Anyway the point of this is that many people have told me their life was significantly changed by working with him on management improvement initiatives (mechanics talking about how he changed the workplace they had been in for years, people who saw that they could contribute more and changed careers, managers that realized how much damage they had done but now were on the right track…). There was obviously a great deal of emotion for many people. And it was largely about applying concepts like Deming’s management system, Toyota Management practices, statistics (yes even that)… and his ability to talk to everyone and make them comfortable (tons of people mentioned this - that this university professor would ask me questions and talk to me like a person, not talk down to me and be interested in my answers and…). As I continue through life I realize that this management improvement stuff really can matter if done right.
I have grown to enjoy maintaining the management improvement resources and other Curious Cat web sites but this is the reason I started and continued these efforts over the years. Today there is a great amount of useful management information online - but for years the pickings were quite slim.
Photo is of Dad and me a few years ago. Related: Quality in the Community: Madison, WI - Statistics for Experimenters - Doing More With Less in the Public Sector: A Progress Report from Madison, Wisconsin - Managing Our Way to Economic Success: Two Untapped Resources - Invest in new management methods not a failing company
Mistakes in Experimental Design and Interpretation
This is an excellent article discussing very common errors in how people use data. We have tendencies that lead us to draw faulty conclusions from data. Given that it is important to understand what common mistakes are made to help us counter the natural tendencies.
Related: Seeing Patterns Where None Exists - Illusions, Optical and Other - Understanding Data - Dangers of Forgetting the Proxy Nature of Data - How to Deal with False Research Findings - descriptive “theory” and normative theory
Jeffrey Pfeffer Testifies to Congress About Evidence-Based Practices:
Great stuff. Read the entire document. via: Bob Sutton’s Work Matters
Related: Evidence-based Management - Illusions - Optical and Other
Books: The Knowing-Doing Gap by Jeffrey Pfeffer and Robert Sutton - Hard Facts, Dangerous Half-Truths And Total Nonsense: Profiting From Evidence-Based Management by Jeffrey Pfeffer and Robert Sutton
Performance Measures and Statistics Course - free course materials from a 2 day training course by Steven Prevette. Topics include: Dr. Deming’s red bead experiment, operational definitions, selecting performance targets, SPC, theory of variation, case studies, control charts, pdsa, pareto charts, histograms…
Related: Quality, SPC and Your Career - articles by Steven Prevette
The Future of Quality Technology: From a Manufacturing to a Knowledge Economy and From Defects to Innovations (pdf) by Soren Bisgaard:
This article does a good job of explaining why “quality/lean…” should not be viewed as just process improvement, and innovation as something separate. I agree, as discussed in: Quality and Innovation. Many quality and lean tools are focused on process improvement. But those tools are part of a system that requires customer focused innovation (including breakthrough innovation).
Also in this issue of the ASQ statistics division newsletter, is the acceptance speech by the most recent Hunter Award (named for my father) winner: Douglas M. Hawkins.
Related: Curious Cat Management Improvement Library - Soren Bisgaard 2002 Hunter Award speech - more articles by Soren Bisgaard - Deming on Innovation - Better and Different - Innovation at Toyota - Six Keys to Building New Markets by Unleashing Disruptive Innovation - Global Manufacturing Data by Country - Manufacturing Jobs Data: USA and China
Never a Dull Day: The Life of an Industrial Statistician by Gerry Hahn.
Gerry Hahn was one of the great applied statisticians of the last 50 years, working at GE for over 45 years. Six sigma has many variants, he is one of those that understood how to apply six sigma well.
In 1991 Dr. Hahn received the Hunter Award from the ASQ Statistics Division (the award is named for my father - John).
Topic: Management Improvement
Statistics Abuse and Me by Jay Mathews:
How can that be? Is it important? First, yes it is important. Effective use of data is an important part of management improvement. Emphasis the effective, not the data. Use of data by itself is not sufficient.
To be effective you need to learn to think about not what is printed on the page but what lies behind the numbers you see. The numbers are just proxies for the real situation. Look beyond the numbers you see to what they mean and understand how the numbers presented may not fully capture the important details you need to consider. (more…)
Quality Quandaries: Six Sigma, Process Drift, Capability Indices, and Feedback Adjustment by George Box and Alberto Luceno. This article is for the more statistically inclined.
The Six Sigma specification makes an allowance of 1.5 standard deviations for process drift. Simple ways in which a major part of such drift can be removed are given. These employ feedback adjustment methods specifically designed for SPC applications. (more…)
Using Design of Experiments as a Process Road Map by Davis Balestracci:
DOE is a tool that is very useful. And while the situations in which DOE is the best tool to use is limited the limited use of DOE is used less than it could be. See more articles on the use of design of Experiments (DOE).
Lead To Succeed (pdf document) by Stephen S. Prevette:
* Succeed as a quality professional by branding yourself and providing a service or product your manager and organization deem worth paying for.
* Lead your manager “your customers” by providing the data they need in a form they can understand.
This is a great article on how to apply quality (Deming, Statistical Process Control, Six Sigma, Lean Manufacturing) ideas and move forward professionally; even when those ideas are not always shared by the organization.
(more…)
Dr. George Box is not as well known in the general management community as his ideas merit (in my biased opinion - photo of Bill Hunter and George Box). He is well know in the statistics field as one of the leading statistical minds. Box on Quality is an excellent book that gathers his essays from his 65th to 80th year. The book has just been issued in paperback (which helps as the hardback was pricey).
While some of the essays are aimed at a reader with an advanced understanding of statistics, many of the articles are aimed at any manager attempting to apply Quality Management principles (SPC, Deming, process improvement, six sigma, etc.). An except from the book provides a table of contents and an introduction.
Some of the articles from the book are available online. I encourage you to take a look at several of the articles and then go ahead and add this book to your prized management resources, if you find them worthwhile.
Curious Cat Management Improvement Blog curiouscat.com 2005-2007 powered by WordPress - management improvement jobs
