How to Manage What You Can’t Measure

In Out of the Crisis, page 121, Dr. Deming wrote:

the most important figures that one needs for management are unknown or unknowable (Lloyd S. Nelson, director of statistical methods for the Nashua corporation), but successful management must nevertheless take account of them.

So what do you do then? I am a strong advocate of Deming’s ideas on management. I see understanding system thinking, psychology, the theory of knowledge and variation as the tools to use when you can’t get precise measures (or when you can).

Even if you can’t measure exactly what you want, you can learn about the area with related data. You are not able to measure the exact benefit of a happy customer but you can get measures that give you evidence of the value and even magnitude. And you can get measures of the costs of dis-satisfied customers. I just mention this to be clear getting data is very useful and most organizations need to focus on gathering sensible data and using it well.

Without precise measures though you must use judgment. Judgment will often be better with an understanding of theory and repeated attempts to test those theories and learn. Understanding variation can be used even if you don’t have control charts and data. Over-reaction to special causes is very common. Even without data, this idea can be used to guide your thoughts.

The danger is that we mistake measures for the thing itself. Measures are a proxy and we need to understand the limitation of the data we use. The main point Deming was making was we can’t just pretend the data we have tells us everything we need to know. We need to think. We need to understand that the data is useful but the limitations need to be remembered.

Human systems involve people. To manage human systems you need to learn about psychology. Paying attention to what research can show about motivation, fear, trust, etc. is important and valuable. It aids management decisions when you can’t get the exact data that you would like. If people are unhappy you can see it. You may also be able to measure aspects of this (increased sick leave, increased turnover…). If people are unhappy they often will not be as pleasant to interact with as people who are happy. You can make judgments about the problems created by internal systems that rob people of joy in work and prevent them from helping customers.

For me the key is to use the Deming’s management system to guide action when you can’t get clear data. We should keep trying to find measures that will help. In my experience even though there are many instances where we can get definite data on exactly what we want we fail to get data that would help guide actions a great deal). Then we need to understand the limitations of the data we can gather. And then we need to continually improve and continually learn.

When you have clear data, Deming’s ideas are also valuable. But when the data is lacking it is even more important to take a systemic approach to making management decisions. Falling back into using the numbers you can get to drive decision making is a recipe for trouble.

Related: Manage what you can’t measureStatistical Engineering Links Statistical Thinking, Methods and Toolsoutcome measures

This entry was posted in Deming, Management, Popular, quote, Systems thinking and tagged , , , , , , , , , . Bookmark the permalink.

12 Responses to How to Manage What You Can’t Measure

  1. Tanmay Vora says:

    Hi John,

    A very timely post and the one I enjoyed reading. With respect to Deming’s Seven Deadly Diseases, I attempted to list the “invisibles” that directly impact business performance. You can check them out at


  2. davidburkus says:

    Tanmay…you beat me too it. I just got done reading your post and went on to this one next. I'd intended on comment on the similarities.

  3. Great post. I’m a fan of Deming too, but he is not a panacea. Drucker, Peters, and others have strongly influenced my management philosophy too. I liked that you pointed out the need to understand psychology, it’s an area where so much new research can shed light that has direct implications for our projects and teams, if we choose to listen and apply.


  4. Glenn Waters says:

    Which group is the linkedin discussion in? When I click on the link all linkedin tells me is that I am not a member of the group. It doesn't tell me the group name. Thx.

  5. John Hunter says:

    The LinkedIn group is:
    The W. Edwards Deming Institute +OFFICIAL GROUP+

    LinkedIn group discussions are much worse for not making the content open and have urls work. Forcing people to register to comment I would be fine with but it is lame that I can’t send the url to people to let them know. LinkedIn has continued to lag behind the social web precisely because of such backwards practices.

    Failing to have urls work kills the social web. Now I guess you can overcome this if you become your own isolated island on the internet with enough users (I guess maybe facebook does this). However, I think it is bad even in those cases. And it is certainly a bad idea in cases where you don’t get the huge user base Facebook does. I suppose LinkedIn feels they can – I don’t think that is wise.

  6. John Hunter says:

    Jose, I agree it is perfectly sensible to bring in ideas of others like Ackoff, Christensen, Drucker… Deming did that all the time. I do see many many failures when organizations try to just skim the surface of management ideas taking a few concepts from here and there. I personally feel you are better off really getting a deep understanding and I think Deming helps this enormously. That said there are many great management organization that don’t explicitly study Deming (though many of these get his ideas, 2nd, 3rd… hand).

  7. Pingback: Curious Cat Management Improvement Blog » Good Process Improvement Practices

  8. Pingback: Unknown and Unknowable Data « The W. Edwards Deming Institute Blog

  9. Pingback: Myth: If You Can’t Measure It, You Can’t Manage It « The W. Edwards Deming Institute Blog

  10. Pingback: The Cleveland Clinic Improvement Model, Part 1 | Lean Blog

  11. Pingback: Undertaking a BPM Initiative and Main Considerations

  12. Pingback: Don’t Expect Short Quotes to Tell the Whole Story | Curious Cat Management Improvement Blog

Comments are closed.