Posts about Data

Manufacturing Outlook and History In the USA and Globally

I write primarily about management improvement on this blog – which makes sense given the title. In the very early days I had more on investing, economic data, science, engineering and travel. Then I created three new blogs (Curious Cat Investment and Economics Blog, Curious Cat Science and Engineering Blog, Curious Cat Travel Photos blog) and that made this blog more focused.

Even so the lines of what fits where can be a bit fuzzy and I continue to write about manufacturing, and health care, with a focus on economic data, occasionally. And that is what I am doing today while touching on management related to manufacturing a bit.

As I have written before the story of manufacturing in the USA, and globally, is greatly increased quality of processes and output as well as greatly improved productivity over the last few decades. Manufacturing output also increased, including in the USA, as I have written consistently for a decade now. For example: (Top 10 Countries for Manufacturing Production from 1980 to 2010.

Still many people have the notion that USA manufacturing has been declining, which hasn’t been true, and certainly isn’t true now (the last couple of years have been especially strong and even the general public seems to realize the idea of the USA losing manufacturing is a myth).

Chart of Manufacturing Output fro 1992 to 2012 - USA, China, Japan and Germany

Based on data from the UN. See my blog post on my economics for more details on the data (to be posted next week).

The chart is impressive and illustrates the point I have been hammering home for years. The USA manufacturing base is growing and far from crumbling (job losses are deceiving as they are global and not an indication of a USA manufacturing decline). China’s manufacturing growth is incredible. China and the USA are far away the top 2 manufacturing countries. Japan and Germany make out the top 4 before a large gap which then is followed by a group of countries that are very close (Korea is 5th with less than half the production of Germany).

Continue reading

The Art of Discovery

Quality and The Art of Discovery by Professor George Box (1990):


Quotes by George Box in the video:

“I think of statistical methods as the use of science to make sense of numbers”

“The scientific method is how we increase the rate at which we find things out.”

“I think the quality revolution is nothing more, or less, than the dramatic expansion of the of scientific problem solving using informed observation and directed experimentation to find out more about the process, the product and the customer.”

“It really amounts to this, if you know more about what it is you are doing then you can do it better and you can do it cheaper.”

“We are talking about involving the whole workforce in the use of the scientific method and retraining our engineers and scientists in a more efficient way to run experiments.”

“Tapping into resources:

  1. Every operating system generates information that can be used to improve it.
  2. Everyone has creativity.
  3. Designed experiments can greatly increase the efficiency of experimentation.

An informed observer and directed experimentation are necessary for the scientific method to be applied. He notes that the control chart is used to notify an informed observer to explain what is special about the conditions when a result falls outside the control limits. When the chart indicates a special cause is likely present (something not part of the normal system) an informed observer should think about what special cause could lead to the result that was measured. And it is important this is done quickly as the ability of the knowledgable observer to determine what is special is much greater the closer in time to the result was created.

The video was posted by Wiley (with the permission of George’s family), Wiley is the publisher of George’s recent autobiography, An Accidental Statistician: The Life and Memories of George E. P. Box, and many of his other books.

Related: Two resources, largely untapped in American organizations, are potential information and employee creativityStatistics for Experimenters (book on directed experimentation by Box, Hunter and Hunter)Highlights from 2009 George Box SpeechIntroductory Videos on Using Design of Experiments to Improve Results (with Stu Hunter)

Management Improvement Blog Carnival #190

The Curious Cat Management Carnival is published twice each month. The posts selected for the carnival focus on the areas of management improvement I have focused on in the Curious Cat Management Improvement Guide since 1996: Deming, evidence based management, systems thinking, respect for people, applied statistics, etc..

photo of George Box, John Hunter and Peter Scholtesphoto of (from right to left) Peter Scholtes, John Hunter and George Box in Madison, Wisconsin at the 2008 Deming Conference
  • George Box (1919 to 2013) by John Hunter – George Box was a very kind, smart, caring and fun person. He was a gifted storyteller and writer. He was also one of the most important statisticians of the last 100 years. He had the ability to present ideas so they were easy to comprehend and appreciate…
  • George Box: A remembrance by Bradley Jones – “His greatest contribution to my life was the wonderful book, Statistics for Experimenters, which he wrote with William G. Hunter and Stu Hunter and published in 1978, the same year he served as president of the American Statistical Association. I remember the excitement I felt on reading the description of how the attainment of knowledge is an endless spiral proceeding alternately from deduction to induction and back. Even now, I recall with pleasure the discussion of the randomization distribution early in the book.”
  • Getting Started with Factorial Design of Experiments by Eston Martz – “When I talk to quality professionals about how they use statistics, one tool they mention again and again is design of experiments, or DOE. I’d never even heard the term before I started getting involved in quality improvement efforts, but now that I’ve learned how it works, I wonder why I didn’t learn about it sooner. If you need to find out how several factors are affecting a process outcome, DOE is the way to go.”
  • Brian Joiner Podcast on Management, Sustainability and the Health Care System – Recently Brian has shifted his focus to the health care system (while maintaining a focus on quality principles and sustainability). “Our health care system is an economic tsunami that is about to overwhelm us if we don’t do something very significant, very soon.”
  • Continue reading

George Box

I would most likely not exist if it were not for George Box. My father took a course from George while my father was a student at Princeton. George agreed to start the Statistics Department at the University of Wisconsin – Madison, and my father followed him to Madison, to be the first PhD student. Dad graduated, and the next year was a professor there, where he and George remained for the rest of their careers.

George died today, he was born in 1919. He recently completed An Accidental Statistician: The Life and Memories of George E. P. Box which is an excellent book that captures his great ability to tell stories. It is a wonderful read for anyone interested in statistics and management improvement or just great stories of an interesting life.

photo of George EP Box

George Box by Brent Nicastro.

George Box was a fantastic statistician. I am not the person to judge, but from what I have read one of the handful of most important applied statisticians of the last 100 years. His contributions are enormous. Several well know statistical methods are known by his name, including:

George was elected a member of the American Academy of Arts and Sciences in 1974 and a Fellow of the Royal Society in 1979. He also served as president of the American Statistics Association in 1978. George is also an honorary member of ASQ.

George was a very kind, caring and fun person. He was a gifted storyteller and writer. He had the ability to present ideas so they were easy to comprehend and appreciate. While his writing was great, seeing him in person added so much more. Growing up I was able to enjoy his stories often, at our house or his. The last time I was in Madison, my brother and I visited with him and again listened to his marvelous stories about Carl Pearson, Ronald Fisher and so much more. He was one those special people that made you very happy whenever you were near him.

George Box, Stuart Hunter and Bill Hunter (my father) wrote what has become a classic text for experimenters in scientific and business circles, Statistics for Experimenters. I am biased but I think this is acknowledged as one of (if not the) most important books on design of experiments.

George also wrote other classic books: Time series analysis: Forecasting and control (1979, with Gwilym Jenkins) and Bayesian inference in statistical analysis. (1973, with George C. Tiao).

George Box and Bill Hunter co-founded the Center for Quality and Productivity Improvement at the University of Wisconsin-Madison in 1984. The Center develops, advances and communicates quality improvement methods and ideas.

The Box Medal for Outstanding Contributions to Industrial Statistics recognizes development and the application of statistical methods in European business and industry in his honor.

All models are wrong but some are useful” is likely his most famous quote. More quotes By George Box

A few selected articles and reports by George Box

Continue reading

Podcast Discussion on Management Matters

I continue to record podcasts as I promote my new book – Management Matters: Building Enterprise Capability. This the second part, of 2, of my podcast with Joe Dager, Business 901: Management Matters to a Curious Cat. The first part featured a discussion of 2 new deadly diseases facing companies.

image of the cover of Managmenet Matters by John Hunter

Management Matters by John Hunter

Download this podcast.

Links to more information on some of the topics I mention in the podcast:

More podcasts: Process Excellence Network Podcast with John HunterBusiness 901 Podcast with John Hunter: Deming’s Management Ideas Today (2012)Leanpub Podcast on Management Matters: Building Enterprise Capability

Leanpub Podcast on My Book – Management Matters: Building Enterprise Capability

image of the cover of Managmenet Matters by John Hunter

Management Matters by John Hunter

I recently was interviewed for a podcast by Len Epp with Leanpub: Leanpub Podcast Interview #9: John Hunter. I hope you enjoy the podcast (download the mp3 of the podcast).

In the podcast we cover quite a bit of ground quickly, so the details are limited (transcript of the interview). These links provide more details on items I mention in the podcast. They are listed below in the same order as they are raised in the podcast:

The last 15 minutes of the podcast I talk about some details of working with Leanpub; I used Leanpub to publish Management Matters. I recommend Leanpub for other authors. They don’t just have lean in their name, they actual apply lean principles (focusing on the value chain, eliminating complexity, customer focus, etc.) to operating Leanpub. It is extremely easy to get started and publish your book.

Leanpub also offers an excellent royalty plan: authors take home 90% of the revenue minus 50 cents per book. They publish without “digital rights management” crippling purchasers use of the books. Buyers have access to pdf, kindle (mobi) and epub (iPad, nook) format books and get access to all updates to the book. All purchases include a 45 day full money back guaranty.

Related: Business 901 Podcast with John Hunter: Deming’s Management Ideas TodayInterviews for Management Matters: Building Enterprise Capability

Special Cause Signal Isn’t Proof A Special Cause Exists

One of my pet peeves is when people say that a point outside the control limits is a special cause. It is not. It is an indication that it likely a special cause exists, and that special cause thinking is the correct strategy to use to seek improvement. But that doesn’t mean there definitely was a special cause – it could be a false signal.

This post relies on an understand of control charts and common and special causes (review these links if you need some additional context).

Similarly, a result that doesn’t signal a special cause (inside the control limits without raising some other flag, say a run of continually increasing points) does not mean a special cause is not present.

The reason control charts are useful is to help us maximize our effectiveness. We are biased toward using special cause thinking when it is not the most effective approach. So the control chart is a good way to keep us focused on common cause thinking for improvement. It is also very useful in flagging when it is time to immediately start using special cause thinking (since timing is key to effective special cause thinking).

However, if there is result that is close to the control limit (but inside – so no special cause is indicated) and the person that works on the process everyday thinks, I noticed x (some special cause) earlier, they should not just ignore that. It very well could be a special cause that, because of other common cause variation, resulted in a data point that didn’t quite reach the special cause signal. Where the dot happened to land (just above or just below the control limit – does not determine if a special cause existed).

The signal is just to help us systemically make the best choice of common cause or special cause thinking. The signal does not define whether a special cause (an assignable cause) exists of not. The control chart tool helps guide us to use the correct type of improvement strategy (common cause or special cause). But it is just a signaling device, it isn’t some arbiter of whether a special cause actually exists.

Continue reading

My New Book: Management Matters

Image of the book cover of Management Matters by John Hunter

Management Matters by John Hunter is now available.

I have a new book in progress: Management Matters. It is now available in “pre-release format” via leanpub. The idea I am experimenting with (supported by leanpub) is pre-publishing the book online. The ebook is available for purchase now, and comes with free access to the updates.

My plan is to continue working on the book for the next few months and have it “release ready” by October, 2012. One of the advantages of this method is that I can incorporate ideas based on feedback from the early readers of the book.

There are several other interesting aspects to publishing in this way. Leanpub allows a suggested retail price, and a minimum price. So I can set a suggested price and a minimum price and the purchaser gets to decide what price to pay (they can even pay over suggested retail price – which does happen). The leanpub model provides nearly all the revenue to the author (unlike traditional models) – the author gets 90% of the price paid, less 50 cents per book (so $8.50 of a $10 purchase).

They provide the book in pdf, mobi (Kindle) and epub (iPad, Nook, etc.) formats. And the books do not have any Digital Rights Management (DRM) entanglements.

Management Matters covers topics familiar to those who have been reading this blog for years. It is an attempt to put in one place the overall management system that is most valuable (which as you know, based on the blog, is largely based upon Dr. Deming’s ideas – which means lean manufacturing are widely covered too).

I hope the book is now in a state where those who are interested would find it useful, but it is in what I consider draft format. I still have much editing to do and content to add.

Leanpub also provides a sample book (where a portion of the content can be downloaded to decide if you want to buy). If you are interested please give it a try and let me know your thoughts.

Introductory Videos on Using Design of Experiments to Improve Results

The video shows Stu Hunter discussing design of experiments in 1966. It might be a bit slow going at first but the full set of videos really does give you a quick overview of the many important aspects of design of experiments including factorial designed experiments, fractional factorial design, blocking and response surface design. It really is quite good, if you find the start too slow for you skip down to the second video and watch it.

My guess is, for those unfamiliar with even the most cursory understanding of design of experiments, the discussion may start moving faster than you can absorb the information. One of the great things about video is you can just pause and give yourself a chance to catch up or repeat a part that you didn’t quite understand. You can also take a look at articles on design of experiments.

I believe design of experiments is an extremely powerful methodology of improvement that is greatly underutilized. Six sigma is the only management improvement program that emphasizes factorial designed experiments.

Related: One factor at a time (OFAT) Versus Factorial DesignsThe purpose of Factorial Designed Experiments

Continue reading

Management Blog Posts From November 2006

I have selected a few great posts from the Curious Cat Management Blog back in November 2006.

  • What Could we do Better? – There are many important ideas to improve management. This is one of the most important tips to aid improvement that I know of: it is easy to do, brings huge benefits and most organizations fail to do it. Ask your customers: “What one thing could we do to improve?”
  • Ackoff’s F-laws: Common Sins of Management presents 13 common sins of management, such as: Managers who don’t know how to measure what they want settle for wanting what they can measure
  • Common Cause Variation – “Every system has variation. Common cause variation is the variation due to the current system. Dr. Deming increased his estimate of variation due to the system (common cause variation) to 97% (earlier in his life he cited figures as low as 80%). Special cause variation is that due to some special (not part of the system) cause.”
  • Sub-Optimize by Interrupting Knowledge Workers – “The general consensus is that the loss from interrupting [software] developers is much greater than for interrupting most other forms of work and therefor a great deal of effort is placed on improving the system to allow developers to focus.”
  • Amazon Innovation – “I believe Amazon uses technology very well. They have done many innovative things. They have been less successful at turning their technology into big profits. But I continue to believe they have a good shot at doing so going forward (and their core business is doing very well I think).” [Amazon announced great sales numbers today, continuing their long term tread. They are also continuing to be very slow to grow profits (CEO, Jeff Bezos remains willing to challenge common practices – such as his willingness to build business and sacrifice current profits)].

Keys to the Effective Use of the PDSA Improvement Cycle

The PDSA improvement cycle was created by Walter Shewhart where Dr. Deming learned about it. An improvement process is now part of many management improvement methods (A3 for lean manufacturing, DMAIC for six sigma and many other modifications). They are fairly similar in many ways. The PDSA cycle (Plan, Do, Study, Act) has a few key pieces that are either absent in most others processes of greatly de-emphasized which is why I prefer it (A3 is my second favorite).

The PDSA cycle is a learning cycle based on experiments. When using the PDSA cycle prediction of the results are important. This is important for several reasons but most notably due to an understanding of the theory of knowledge. We will learn much more if we write down our prediction. Otherwise we often just think (after the fact); yeah that is pretty much what I expected (even if it wasn’t). Also we often fail to think specifically enough at the start to even have a prediction. Forcing yourself to make a prediction gets you to think more carefully up front and can help you set better experiments.

An organization using PDSA well will turn the PDSA cycle several times on any topic and do so quickly. In a 3 month period turning it 5 times might be good. Often those organizations that struggle will only turn it once (if they are lucky and even reach the study stage). The biggest reason for effective PDSA cycles taking a bit longer is wanting more data than 2 weeks provides. Still it is better to turn it several times will less data – allowing yourself to learn and adjust than taking one long turn.

The plan stage may well take 80% (or even more) of the effort on the first turn of the PDSA cycle in a new series. The Do stage may well take 80% of of the time – it usually doesn’t take much effort (to just collect a bit of extra data) but it may take time for that data to be ready to collect. In the 2nd, 3rd… turns of the PDSA cycle the Plan stage often takes very little time. Basically you are just adjusting a bit from the first time and then moving forward to gather more data. Occasionally you may learn you missed some very important ideas up front; then the plan stage may again take some time (normally if you radically change your plans).

Remember to think of Do as doing-the-experiment. If you are “doing” a bunch of work (not running an experiment and collecting data) that probably isn’t “do” in the PDSA sense.

Study should not take much time. The plan should have already have laid out what data is important and an expectation of what results will be achieved and provide a good idea on next steps. Only if you are surprised (or in the not very common case that you really have no idea what should come next until you experiment) will the study phase take long.

Continue reading

2011 Management Blog Roundup: Stats Made Easy

The 4th Annual Management blog roundup is coming to a close soon. This is my 3rd and final review post looking back at 2001, the previous two posts looked at: Gemba Panta Rei and the Lean Six Sigma Blog.

I have special affinity for the use of statistics to understand and improve. I imaging it is both genetic and psychological. My father was a statistician and I have found memories of applying statistical thinking to understand a result or system. I also am comfortable with numbers, and like most people enjoy working with things I have an affinity for.

photo of Mark Anderson

Mark Anderson

Mark Anderson’s Stats Made Easy blog brings statistical thinking to managers. And this is not an easy thing to do, as one of his posts shows, we have an ability to ignore data we don’t want to know. Wrong more often than right but never in doubt: “Kahneman examined the illusion of skill in a group of investment advisors who competed for annual performance bonuses. He found zero correlation on year-to-year rankings, thus the firm was simply rewarding luck. What I find most interesting is his observation that even when confronted with irrefutable evidence of misplaced confidence in one’s own ability to prognosticate, most people just carry on with the same level of self-assurance.”

That actually practice of experimentation (PDSA…) needs improvement. Too often the iteration component is entirely missing (only one experiment is done). That is likely partially a result another big problem: the experiments are not nearly short enough. Mark offered very wise advice on the Strategy of experimentation: Break it into a series of smaller stages. “The rule-of-thumb I worked from as a process development engineer is not to put more than 25% of your budget into the first experiment, thus allowing the chance to adapt as you work through the project (or abandon it altogether).” And note that, abandon it altogether option. Don’t just proceed with a plan if what you learn makes that option unwise: too often we act based on expectations rather than evidence.

In Why coaches regress to be mean, Mark explained the problem with reacting to common cause variation and “learning” that it helped to do so. “A case in point is the flight instructor who lavishes praise on a training-pilot who makes a lucky landing. Naturally the next result is not so good. Later the pilot bounces in very badly — again purely by chance (a gust of wind). The instructor roars disapproval. That seems to do the trick — the next landing is much smoother.” When you ascribe special causation to common cause variation you often confirm your own biases.

Mark’s blog doesn’t mention six sigma by name in his 2011 posts but the statistical thinking expressed throughout the year make this a must for those working in six sigma programs.

Related: 2009 Curious Cat Management Blog Carnival2010 Management Blog Review: Software, Manufacturing and Leadership

Steve Jobs Discussing Customer Focus at NeXT

Video from 1991 when Steve Jobs was at NeXT. Even with the customer focus however, NeXT failed. But this does show the difficulty in how to truly apply customer focus. You have to be creative. You have examine data. You have to really understand how your customers use your products or services (go to the gemba). You have to speculate about the future. The video is also great evidence of providing insight to all employees of the current thinking of executives.

Related: Sometimes Micro-managing Works (Jobs)Delighting CustomersWhat Job Does Your Product Do?

Annual Performance Reviews Are Obsolete

Sam Goodner, the CEO of Catapult Systems, wrote about his decision to eliminate the annual performance appraisal.

the most critical flaw of our old process was that the feedback itself was too infrequent and too far removed from the actual behavior to have any measurable impact on employee performance.

I decided to completely eliminate of our annual performance review process and replace it with a real-time performance feedback dashboard.”

I think this is a good move in the right direction. I personally think it is a mistake to make the measures focused on the person. There should be performance dashboards (with in-process and outcome measures) that provide insight into the state of the processes in the company. Let those working in those processes see, in real time, the situation, weaknesses, strengths… and take action as appropriate (short term quick fixes, longer term focus on areas for significant improvement…). It could be the company is doing this, the quick blog post is hardly a comprehensive look at their strategies. It does provide some interesting ideas.

I also worry about making too much of the feedback without an understanding of variation (and the “performance” results attributed to people due merely to variation) and systems thinking. I applaud the leadership to make a change and the creative attempt, I just also worry a bit about how this would work in many organizations. But that is not really what matters. What matters is how it works for their organization, and I certainly believe this could work well in the right organization.

Related: Righter Performance AppraisalWhen Performance-related Pay BackfiresThe Defect Black Marketarticles, books, posts on performance appraisal

A Theory of a System for Educators and Managers

Excerpts from The Deming Library Volume XXI, Dr. W. Edwards Deming, Dr. Russell Ackoff and David Langford demonstrate that educators can begin a quality transformation by developing an understanding of the properties and powers of systems-oriented thinking. You can order the entire video, as well as the rest of The Deming Library.

Great stuff! If you enjoy this blog (the Curious Cat Management Improvement Blog), you definitely should watch this webcast. This video has some great insight into education, learning and systems thinking. It also provides a good explanation of systems thinking compared to analysis. Dr. Ackoff: “You cannot explain the behavior of a system by analysis.” “The performance of the whole is never the sum of the performance of the parts taken separately: but it’s the product of their interactions. Therefore, the basic managerial idea introduced by systems thinking is that to manage a system effectively you must focus on the interactions of the parts rather than their behavior taken separately.”

Dr. Deming: “You may reduce defects to zero and go out of business.”

Dr. Ackoff: “Most discussion of education assume that the best way to learn a subject is to have it taught to you. That’s nonsense… Teaching is a wonderful way to learn. Therefore if we want people to learn we have to make them teach.” If you want more on this see David Langford’s work which provides great advice on how to improve learning and education.

Related: Dr. Deming Webcast on the 5 Deadly DiseasesAn Introduction to Deming’s Management Ideas by Peter ScholtesHow to Manage What You Can’t MeasureMarissa Mayer Webcast on Google InnovationTraffic Congestion and a Non-Solution

Managing Our Way to Economic Success

From Managing Our Way to Economic Success, Two Untapped Resources by William G. Hunter, my father. Written in 1986, but still plenty relevant. We have made some good progress, but there is much more to do: we have barely started adopting these ideas systemically.

there are two enormously valuable untapped resources in many companies: potential information and employee creativity. The two are connected. One of the best ways to generate potential information to turn it into kinetic information that can produce tangible results is to train all employees in some of the simple, effective ways to do this. Rely on their desire to do a good job, to contribute, to be recognized, to be a real part of the organization. They want to be treated like responsible human beings, not like unthinking automatons.

W. Edwards Deming has illustrated one of the troubles with U.S. industry in terms of making toast. He says, “Let’s play American industry. I’ll burn. You scrape.” Use of statistical tools, however, allows you to reduce waste, scrap, rework, and machine downtime. It costs just as much to make defective products as it does to make good products. Eliminate defects and other things that cause inefficiencies, and you reduce costs, increase quality, and raise productivity. Note that quality and productivity are not trade-offs. They increase together.

Potential information surrounds all industrial processes. Statistical techniques, many of which are simple yet powerful, are tools that employees can use to tap and exploit this potential information so that increasingly higher levels of productivity, quality, and innovation can be attained. Engaging the brains as well as the brawn of employees in this way improves morale and participation…and profits.

What is called for is constant, never-ending improvement of all processes in the organization. What management needs, too, is constant, never-ending improvement of ideas.

Related: William Hunter, articles and booksInvest in New Management Methods Not a Failing CompanyThe Importance of Management ImprovementStatistics for Experimenters

How to Manage What You Can’t Measure

In Out of the Crisis, page 121, Dr. Deming wrote:

the most important figures that one needs for management are unknown or unknowable (Lloyd S. Nelson, director of statistical methods for the Nashua corporation), but successful management must nevertheless take account of them.

So what do you do then? I am a strong advocate of Deming’s ideas on management. I see understanding system thinking, psychology, the theory of knowledge and variation as the tools to use when you can’t get precise measures (or when you can).

Even if you can’t measure exactly what you want, you can learn about the area with related data. You are not able to measure the exact benefit of a happy customer but you can get measures that give you evidence of the value and even magnitude. And you can get measures of the costs of dis-satisfied customers. I just mention this to be clear getting data is very useful and most organizations need to focus on gathering sensible data and using it well.

Without precise measure though you have to use judgment. Judgment will often be better with an understanding of theory and repeated attempts to test those theories and learn. Understanding variation can be used even if you don’t have control charts and data. Over-reaction to special causes is very common. Even without data, this idea can be used to guide your thoughts.

The danger is that we mistake measures for the thing itself. Measures are a proxy and we need to understand the limitation of the data we use. The main point Deming was making was we can’t just pretend the data we have tells us everything we need to know. We need to think. We need to understand that the data is useful but the limitations need to be remembered.

Human systems involve people. To manage human systems you need to learn about psychology. Paying attention to what research can show about motivation, fear, trust, etc. is important and valuable. It aids management decisions when you can’t get the exact data that you would like. If people are unhappy you can see it. You may also be able to measure aspects of this (increased sick leave, increased turnover…). If people are unhappy they often will not be as pleasant to interact with as people who are happy. You can make judgments about the problems created by internal systems that rob people of joy in work and prevent them from helping customers.

For me the key is to use the Deming’s management system to guide action when you can’t get clear data. We should keep trying to find measures that will help. In my experience even though there are many instances where we can get definite data on exactly what we want we fail to get data that would help guide actions a great deal). Then we need to understand the limitations of the data we can gather. And then we need to continually improve and continually learn.

When you have clear data, Deming’s ideas are also valuable. But when the data is lacking it is even more important to take a systemic approach to making management decisions. Falling back into using the numbers you can get to drive decision making is a recipe for trouble.

LinkedIn discussion on the topic

Related: Manage what you can’t measureStatistical Engineering Links Statistical Thinking, Methods and Toolsoutcome measures

Actionable Metrics

Metrics are valuable when they are actionable. Think about what will be done if certain results are shown by the data. If you can’t think of actions you would take, it may be that metric is not worth tracking.

Metrics should be operationally defined so that the data is collected properly. Without operationally definitions data collected by more than one person will often include measurement error (in this case, the resulting data showing the results of different people measuring different things but calling the result the same thing).

And without operational definitions those using the resulting data may well mis-interpret what it is saying. Often data is presented without an operational definition and people think the data is saying something that it is not. I find most often when people say statistics lie it is really that they made an incorrect assumption about what the data said – which most often was because they didn’t understand the operational definition of the data. Data can’t lie. People can. And people can intentionally mislead with data. But far more often people unintentionally mislead with data that is misunderstood (often this is due to failure to operationally define the data).

In response to: Metrics Manifesto: Raising the Standard for Metrics

Related: Outcome MeasuresEvidence-based ManagementMetrics and Software DevelopmentDistorting the System (due to misunderstanding metrics)Manage what you can’t measure

Problems With Student Evaluations as Measures of Teacher Performance

Dr. Deming was, among other things a professor. He found the evaluation of professors by students an unimportant (and often counterproductive measure) – used in some places for awards and performance appraisal. He said for such a measure to be useful it should survey students 20 years later to see which professors made a difference to the students. Here is an interesting paper that explored some of these ideas. Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors by Scott E. Carrell, University of California, Davis and National Bureau of Economic Research; and James E. West, U.S. Air Force Academy:

our results indicate that professors who excel at promoting contemporaneous student achievement, on average, harm the subsequent performance of their students in more advanced classes. Academic rank, teaching experience, and terminal degree status of professors are negatively correlated with contemporaneous value‐added but positively correlated with follow‐on course value‐added. Hence, students of less experienced instructors who do not possess a doctorate perform significantly better in the contemporaneous course but perform worse in the follow‐on related curriculum.

Student evaluations are positively correlated with contemporaneous professor value‐added and negatively correlated with follow‐on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value‐added in follow‐on courses). Since many U.S. colleges and universities use student evaluations as a measurement of teaching quality for academic promotion and tenure decisions, this latter finding draws into question the value and accuracy of this practice.

These findings have broad implications for how students should be assessed and teacher quality measured.

Related: Applying Lean Tools to University CoursesK-12 Educational ReformImproving Education with Deming’s IdeasLearning, Systems and ImprovementHow We Know What We Know

Taxes per Person by Country

I think that the idea that data lies is false, and that such a notion is commonly held a sign of lazy intellect. You can present data in different ways to focus on different aspects of a system. And you can make faulty assumptions based on data you look at.

It is true someone can just provide false data, that is an issue you have to consider when drawing conclusions from data. But often people just don’t think about what the data is really saying. Most often when people say data lies they just were misled because they didn’t think about what the data actually showed. When you examine data provided by someone else you need to make sure you understand what it is actually saying and if they are trying to support their position you may be wise to be clear they are not misleading you with their presentation of the data.

Here is some data from Greg Mankiw’s Blog. He wants to make his point that the USA is taxed more on par with Europe than some believe because he want to reduce current taxes. So he shows that while taxes as a percent of economic activity is low in the USA taxes per person is comparable to Europe.

Taxes/GDP x GDP/Person = Taxes/Person

France .461 x 33,744 = $15,556

Germany .406 x 34,219 = $13,893

UK .390 x 35,165 = $13,714

US .282 x 46,443 = $13,097

Canada .334 x 38,290 = $12,789

Italy .426 x 29,290 = $12,478

Spain .373 x 29,527 = $11,014

Japan .274 x 32,817 = $8,992

The USA is the 2nd lowest for percent of GDP taxes 28.2% v 27.4% for Japan. But in taxes per person toward the middle of the pack. France which has 46% taxes/GDP totals $15,556 in tax per person compared to $13,097 for the USA. Both measures of taxes are useful to know, in my opinion. Neither lies. Both have merit in providing a understanding of the system (the economies of countries).

Related: Fooled by RandomnessSimpson’s ParadoxMistakes in Experimental Design and InterpretationGovernment Debt as Percentage of GDP 1990-2008 by CountryCommunicating with the Visual Display of DataIllusion of Explanatory Depth

  • Recent Trackbacks

  • Comments