|
|
|
|
|
|
posts relating to data, data based decisions, statistics, SPC, data quality, data analysis, data collection... It is critical to understand common and special cause variation.
Recommended posts: Data is Only a Proxy - Targets Distorting the System - Measurement and Data Collection - Evidence-based Management - Visible Data - Understanding Data
Related: Operational definition - Control chart
Secret of Googlenomics: Data-Fueled Recipe Brews Profitability
Google continues to make bold moves putting faith in their ability to find innovative solutions that others reject as impossible. It is a challenging but interesting path to success, for them, at least.
Related: Google Should Stay True to Their Management Practices - Google’s Answer to Filling Jobs Is an Algorithm - The Google Way: Give Engineers Room - Google Website Optimizer - Google: Experiment Quickly and Often - posts on innovation in management
Revealed Preference: the preference consumers display by their action, in contrast to what they may say they prefer. While surveys may be useful people often say they will do one thing and actually when given the choice to do so, don’t.
Normally what matters is not what people say they want but what they actually will choose. For that reason revealed preference is a better measure than stated preference. Stated preference is often used as a proxy for actual preference (which may be fine) but it is important to understand that it is just a proxy for actual preference.
See more explanations from the Curious Cat Management Dictionary.
Related: Packaging Improvement - All Models Are Wrong But Some Are Useful - Dangers of Forgetting the Proxy Nature of Data - Confirmation Bias - Be Careful What You Measure
Dr. Deming long ago stated in his 14 obligations of management: “Eliminate numerical goals, numerical quotas and management by objectives.” I think he was right then, and is right now. A goal can help set the scope of the effort. If you are aiming for 2% improvement different strategies may make sense than if you are aiming at 50% improvement. But that type of use is rare. The problem with goals is what actually happens in organizations. They create serious systemic problems and should be avoided (other than in setting the scope). They are deeply ingrained in the way many people think, but we would be better if we could eliminate the use of goals, as they are used now (mainly as arbitrary numerical goals).
Ready, Aim … Fail, Why setting goals can backfire
Today, as the economic situation upends millions of lives, it is also forcing the reexamination of millions of goals - not only the revenue targets of battered firms, but the career aims of workers and students, and even the ambitions of the newly installed administration. And while it never feels good to give up on a goal, it may be a good time to ask which of the goals we had set for ourselves were things we really needed to achieve, and which were things we only thought we should - and what the difference has been costing us.
Related: Measuring and Managing Performance in Organizations - Arbitrary Rules Don’t Work - The Defect Black Market - Goals can Distract from Improvement - Be Careful What You Measure
Dr. Deming used the red bead experiment to present a view into management practices and his management philosophy. The experiment provides insight into all four aspects of Dr. Deming’s management system: understanding variation, understanding psychology, systems thinking and the theory of knowledge.
Red Bead Experiment by Steve Prevette
Related: Fooled by Randomness - Performance Measures and Statistics Course - Performance without Appraisal - Exploring Deming’s Management Ideas - Eliminate Slogans
Lynda Finn, President of Statistical Insight, has written an article on how to create a control chart for seasonal or trending data (where there is an underlying structural variation in the data). Essentially you need to account for the structural variation to create the control limits for the control chart. She also provides a Minitab project file. Both are available for download from the Curious Cat Management Improvement Library.
Related: Control Charts in Health Care - Common Cause Variation - Managing with Control Charts - Measurement and Data Collection - Fourth Generation Management
From the excellent xkcd comic.
Related: Correlation is Not Causation - Does the Data Deluge Make the Scientific Method Obsolete? - Understanding Data - Theory of Knowledge - What Makes Scientists Different 
- Dangers of Forgetting the Proxy Nature of Data - Seeing Patterns Where None Exists
Statistics for Experimenters, second edition, by George E. P. Box, J. Stuart Hunter and William G. Hunter (my father) is now available in Spanish.
Read a bit more can find a bit more on the Spanish edition, in Spanish. Estadística para Investigadores Diseño, innovación y descubrimiento Segunda edición.
Statistics for Experimenters - Second Edition:
Book available via Editorial Reverte
Related: Statistics for Experimenters Review - Correlation is Not Causation - Statistics for Experimenters Data - posts on design of experiments
What’s the Value of a Big Bonus? by Dan Ariely
When I recently presented these results to a group of banking executives, they assured me that their own work and that of their employees would not follow this pattern. (I pointed out that with the right research budget, and their participation, we could examine this assertion. They weren’t that interested.)
This is an interesting look at an effect of bonuses. We all know monetary bonuses can influence behavior. The problem is the type of behaviors that result. Huge bonuses, for example, create huge incentives to risk the future of the company for the chance at a huge bonus for the executive. Extrinsic motivation leads to many problems.
Problems with bonuses: Losses Covered Up to Protect Bonuses - “Pay for Performance” is a Bad Idea - Problems with Bonuses - Book: Punished By Rewards: The Trouble With Gold Stars, Incentive Plans, A’s, Praise, and Other Bribes by Alfie Kohn - posts on executive pay
I continue to tilt at the robber barron CEO pay packages (2007 post on CEO pay abuses).
| 2007 pay rank |
Company | CEO | Pay | 5 Year Pay | CEO % of 2007 Earnings | |
|---|---|---|---|---|---|---|
| 1 | Apple | Steve Jobs | $646,600,000 | $650,170,000 |
| 18.5% |
| 2 | Occidental Petroleum | Ray Irani | $321,640,000 | $509,530,000 |
| 5.9% |
| 3 | IAC | Barry Diller | $295,140,000 | $512,270,000 |
| Company Lost Money |
| 4 | Fidelity National Financial | William Folley | $179,560,000 | NA |
| 138.4% |
| 5 | Yahoo! | Terry Semel | $174,200,000 | $432,490,000 |
| 26.4% |
| 7 | Countrywide Financial | Angelo Mozilo | $141,980,000 | $295,730,000 |
| Company Lost Money |
| 13 | XTO Energy | Bob Simpson | $72,270,000 | $215,280,000 |
| 4.2% |
Data via: Forbes CEO Compensation (Total compensation for each chief executive includes the following: salary and bonuses; other compensation, such as vested restricted stock grants, LTIP payouts and perks; and stock gains, the value realized by exercising stock options.) and Google Finance (using 2007 earnings - Countrywide from SEC). I realize this chart could be improved by spending more time (the effect of stock options exercised in one year distorts things a bit but the excess are so massively huge that the clarity of the data does not need to be very precise).
(more…)
The updated data from the United Nations on manufacturing output by country clearly shows the USA remains by far the largest manufacturer in the world. UN Data, in billions of current US dollars:
| Country | 1990 | 1995 | 2000 | 2005 | 2006 | 2007 |
|---|---|---|---|---|---|---|
| USA | 1,041 | 1,289 | 1,543 | 1,663 | 1,700 | 1,831 |
| China | 143 | 299 | 484 | 734 | 891 | 1,106 |
| Japan | 804 | 1,209 | 1.034 | 954 | 934 | 926 |
| Germany | 438 | 517 | 392 | 566 | 595 | 670 |
| Russian Federation | 211 | 104 | 73 | 222 | 281 | 362 |
| Italy | 240 | 226 | 206 | 289 | 299 | 345 |
| United Kingdom | 207 | 219 | 228 | 269 | 303 | 342 |
| France | 224 | 259 | 190 | 249 | 248 | 296 |
| Korea | 65 | 129 | 134 | 200 | 220 | 241 |
| Canada | 92 | 100 | 129 | 177 | 195 | 218 |
See manufacturing data for more countries.
The USA’s share of the manufacturing output of the countries that manufactured over $200 billion in 2007 (the 12 countries on the top of the chart above) in 1990 was 28%, 1995 28%, 2000 33%, 2005 30%, 2006 28%, 2007 27%. China’s share has grown from 4% in 1990, 1995 7%, 2000 11%, 2005 13%, 2006 15%, 2007 16%.
Total manufacturing output in the USA was up 76% in 2007 from the 1990 level. Japan, the second largest manufacturer in 1990, and third today, has increased output 15% (the lowest of the top 12, France is next lowest at 32%) while China is up an amazing 673% (Korea is next at an increase of 271%).
(more…)
The End of Theory: The Data Deluge Makes the Scientific Method Obsolete by Chris Anderson
So proclaimed statistician George Box 30 years ago, and he was right. But what choice did we have? Only models, from cosmological equations to theories of human behavior, seemed to be able to consistently, if imperfectly, explain the world around us. Until now. Today companies like Google, which have grown up in an era of massively abundant data, don’t have to settle for wrong models. Indeed, they don’t have to settle for models at all.
…
Speaking at the O’Reilly Emerging Technology Conference this past March, Peter Norvig, Google’s research director, offered an update to George Box’s maxim: “All models are wrong, and increasingly you can succeed without them.”
…
There is now a better way. Petabytes allow us to say: “Correlation is enough.” We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.
see update, below. Norvig was misquoted, he agrees with Box’s maxim
I must say I am not at all convinced that a new method without theory ready to supplant the existing scientific method. Now I can’t find peter Norvig’s exact words online (come on Google - organize all the world’s information for me please). If he said that using massive stores of data to make discoveries in new ways radically changing how we can learn and create useful systems, that I believe. I do enjoy the idea of trying radical new ways of viewing what is possible.
Practice Makes Perfect: How Billions of Examples Lead to Better Models (summary of his talk on the conference web site):
Related: Will the Data Deluge Makes the Scientific Method Obsolete? - Pragmatism and Management Knowledge - Data Based Decision Making at Google - Seeing Patterns Where None Exists - Manage what you can’t measure - Data Based Blathering - Understanding Data - Webcast on Google Innovation
(more…)
Malcolm Gladwell presented at the New Yorker conference on the Challenge of Hiring in the Modern World. As usually, he provides some great thoughts. I wrote on Hiring the Right Workers
Malcolm Gladwell doesn’t use the same language but I think he says many of the same ideas: “Insisting on managing by the numbers even when the most important figures are unknown and maybe unknowable.” etc. This idea he frames as a mismatch problem.
Related: Hiring: Silicon Valley Style - People are Our Most Important Asset - Malcolm Gladwell Synchronicity - Hiring, Does College Matter? - Interviewing and Hiring Programmers - Gladwell (and Drucker) on Pensions
Amazing Stat: California Uses More Gas than China:
That’s according to the California Energy Commission’s State Alternative Fuels Plan, which was posted online last Christmas Eve (pdf). The whole report makes for some fascinating reading because it’s a blueprint for a low-carbon and renewable transportation fuel future. The dominant takeaway: it ain’t going to be easy.
One more choice statistic: gasoline usage in California has increased 50 percent, that’s 10 6.7 billion gallons, since 1988.
…
But China’s oil thirst is growing — to almost 20 billion gallons in 2007 — and perhaps as early as this year, China’s 1.3 billion people will overtake California’s 37 million people in total gasoline and diesel usage.
Interesting data. The Curious Cat Economics Blog recently posted on the top oil consuming countries.
Related: Car Powered Using Compressed Air - Failure to Increase Gas Tax - Curious Cat Science and Engineering Blog - Energy posts
An outcome measure is used to measure the success of a system. For example, the outcome measure could be the percentage of people who do not get polio (the result). An output measure, for example, would be the number of people vaccinated with the polio vaccine (the output). Often we measure inputs (amount of money spent) or outputs (number of people vaccinated). They are usually easy to measure but obviously less valuable proxies for what the objective of the system (reducing the incidence of polio).
You should have all these types of measures but outcome measures are most likely to be missing so special care should be taken to make sure you are using them. It is important to define good outcome measures to use in determining the success of systems, and in determining the whether improvement projects actually result in improved outcomes.
In-process measures can be valuable in providing actionable information sooner than the outcome measure would allow action. In the polio example, an in process measure example could be % of vaccination by the time a babies is 18 months old. And looking across a country say it might well make sense to stratify the data to see if certain areas were doing poorly on this measure. If so that might be where to focus improvement. You don’t need to wait until people not vaccinated start contracting polio (which will likely be delayed for years after the system starts to have processes fail, in this example) to then notice the problem and then react.
Waiting for the outcome measure to point to a problem in this case (and in many cases) is far too late for process improvement. So process measures are needed to aid in managing the system and reacting to process results, before those processes create poor results (and can be seen as poor outcome measures). More on outcome measures.
Related: Operational Definition - tampering - management improvement web search - Measuring and Managing Performance in Organizations - Data is a Proxy - posts on managing using data
The recent drastic reductions again emphasize (once again) that changes in the federal funds rate are not correlated with changes in the 30 year fixed mortgage rate. In the last 4 months the discount rate has been reduced nearly 200 basis points, while 30 year fixed mortgage rates have fallen 18 basis points.
I have update my article showing the historical comparison of 30 year fixed mortgage rates and the federal funds rate. The chart shows the federal funds rate and the 30 year fixed rate mortgage rate from January 2000 through April 2008 (for more details see the article).
It is surprising to me how incorrect beliefs about what predicted federal funds rate cuts will mean for mortgage rates are so common when the data makes it clear that such beliefs (I should wait because the fed is going to cut the fed funds rate and so if I wait my mortgage rate will be lower) are unwarranted. But I should not be this type of behavior goes on all the time, inside organization and outside them.
(more…)
Well, this doesn’t sound very well thought out. Bonuses often distort behavior. Dr. Deming was not against such targets and bonuses because he thought they would not result in bugs being fixed: Dr. Deming on the problems with targets or goals. It is a question of how that will happen. The system being distorted is the most likely result of any such system.
Everyone worked even harder on the third day. On the fourth day, however, the well had started to dry up. The testers ran, re-ran, and re-ran again the test cases, but they could only find a handful of issues. The developers strained the issue-tracking system, constantly reloading the “unassigned bugs” page and rushing to self-assign anything that appeared.
And then something strange happened at lunch. Instead of going out to eat with his usual teammates, one of the developers went out with a tester. Soon after, another developer went out with another tester. Within a few minutes, almost all of the developers had paired up with testers.
As the developers returned from lunch, they immediately got to work. Instead of scavenging for newly found bugs, they worked on “code refactoring” and new functionality. And as soon as they deployed their changes, testers found bugs — minor, obscure bugs that a developer could easily overlook. And just as quickly as testers found bugs, the developers were able to fix them and re-deploy. By the end of the day, developers and testers had earned an average of $120.
This is a good post on the systemic drivers of complex processes, take the time to read the whole post. I have a bias is against off the shelf software as it often ends up forcing the process to be designed around the software. And with the amazing power and relative ease of web based applications creating solutions that are specifically designed to the organization are often relatively easy. And yet, as indicated in this article there is often a strong bias in the other direction for buying off the shelf software because it is cheaper and/or faster.
Of course, the decision in each case must be weighed to determine the benefits and cost of the various alternatives. Just remember, if you decide you want simple and flexible, to have your decisions reflect that. I enjoy a telling quote from a software vendor on Toyota’s IT expectations: “it demands that the software or technology be flexible and adapt, often by customizing the code, to its business processes, and not the other way around.” They are right.
Related: Agile Software Development - Complicating Simplicity - Joy in Software Development
The Rationale of Scientific Experimentation by John Dowd explains the value of designed experiments.
In addition to their efficiency, factorial designs also offer the only method of detecting interactions through experimentation. Because numerous factors can be combined in the same series of experimental runs, the interactions can be detected and the nature of their effects can be evaluated when they are present.
The paper also explains analytic and enumerative studies. Dr. Deming stressed the importance of understanding the distinction between the two.
Related: management improvement articles - Design of Experiments articles - Statistics for Experimenters - search statistical management improvement sites - Using Design of Experiments
Another interesting experiment from Google: Using Prediction Markets to Track Information Flows: Evidence from Google
Interesting paper. I would guess most readers of this blog won’t be able to apply prediction markets to there workplace in the short term but never-the-less I find the paper interesting.
Related: Management is Prediction - Google Experiments Quickly and Often - Secrets of the World’s Best Companies
I am reading a fascinating book by Jessica Snyder Sachs: Good Germs, Bad Germs. From page 108:
This is a great example of a positive special cause. How would you identify this? First you would have to stratify the data. It also shows that sometimes looking at the who is important (the problem is just that we far too often look at who instead of the system so at times some get the idea that it is not ok to stratify data based on who - it is just be careful because we often do that when it is not the right approach and we can get fooled by random variation into thinking there is a cause - see the red bead experiment for an example); that it is possible to stratify the data by person to good effect.
The following 20 pages in the book are littered with very interesting details many of which tie to thinking systemically and the perils of optimizing part of the system (both when considering the system to be one person and also when viewing it as society).
I have recently taken to reading more and more about viruses, bacteria, cells, microbiology etc.: it is fascinating stuff.
Related: Science Books by topic - Data Can’t Lie - Understanding Data
Curious Cat Management Improvement Blog curiouscat.com 2005-2006 powered by WordPress - Curious Cat Alumni Directory
