How to Manage What You Can’t Measure

Posted on August 4, 2010  Comments (11)

In Out of the Crisis, page 121, Dr. Deming wrote:

the most important figures that one needs for management are unknown or unknowable (Lloyd S. Nelson, director of statistical methods for the Nashua corporation), but successful management must nevertheless take account of them.

So what do you do then? I am a strong advocate of Deming’s ideas on management. I see understanding system thinking, psychology, the theory of knowledge and variation as the tools to use when you can’t get precise measures (or when you can).

Even if you can’t measure exactly what you want, you can learn about the area with related data. You are not able to measure the exact benefit of a happy customer but you can get measures that give you evidence of the value and even magnitude. And you can get measures of the costs of dis-satisfied customers. I just mention this to be clear getting data is very useful and most organizations need to focus on gathering sensible data and using it well.

Without precise measure though you have to use judgment. Judgment will often be better with an understanding of theory and repeated attempts to test those theories and learn. Understanding variation can be used even if you don’t have control charts and data. Over-reaction to special causes is very common. Even without data, this idea can be used to guide your thoughts.

The danger is that we mistake measures for the thing itself. Measures are a proxy and we need to understand the limitation of the data we use. The main point Deming was making was we can’t just pretend the data we have tells us everything we need to know. We need to think. We need to understand that the data is useful but the limitations need to be remembered.

Human systems involve people. To manage human systems you need to learn about psychology. Paying attention to what research can show about motivation, fear, trust, etc. is important and valuable. It aids management decisions when you can’t get the exact data that you would like. If people are unhappy you can see it. You may also be able to measure aspects of this (increased sick leave, increased turnover…). If people are unhappy they often will not be as pleasant to interact with as people who are happy. You can make judgments about the problems created by internal systems that rob people of joy in work and prevent them from helping customers.

For me the key is to use the Deming’s management system to guide action when you can’t get clear data. We should keep trying to find measures that will help. In my experience even though there are many instances where we can get definite data on exactly what we want we fail to get data that would help guide actions a great deal). Then we need to understand the limitations of the data we can gather. And then we need to continually improve and continually learn.

When you have clear data, Deming’s ideas are also valuable. But when the data is lacking it is even more important to take a systemic approach to making management decisions. Falling back into using the numbers you can get to drive decision making is a recipe for trouble.

Related: Manage what you can’t measureStatistical Engineering Links Statistical Thinking, Methods and Toolsoutcome measures

11 Responses to “How to Manage What You Can’t Measure”

  1. Tanmay Vora
    August 4th, 2010 @ 12:05 pm

    Hi John,

    A very timely post and the one I enjoyed reading. With respect to Deming’s Seven Deadly Diseases, I attempted to list the “invisibles” that directly impact business performance. You can check them out at http://www.qaspire.com/blog.

    Best,
    Tanmay

  2. davidburkus
    August 4th, 2010 @ 5:12 pm

    Tanmay…you beat me too it. I just got done reading your post and went on to this one next. I'd intended on comment on the similarities.

  3. Josh Nankivel
    August 5th, 2010 @ 12:37 am

    Great post. I’m a fan of Deming too, but he is not a panacea. Drucker, Peters, and others have strongly influenced my management philosophy too. I liked that you pointed out the need to understand psychology, it’s an area where so much new research can shed light that has direct implications for our projects and teams, if we choose to listen and apply.

    -Josh

  4. Glenn Waters
    August 5th, 2010 @ 7:13 pm

    Which group is the linkedin discussion in? When I click on the link all linkedin tells me is that I am not a member of the group. It doesn't tell me the group name. Thx.

  5. John Hunter
    August 7th, 2010 @ 12:44 pm

    The LinkedIn group is:
    The W. Edwards Deming Institute +OFFICIAL GROUP+

    LinkedIn group discussions are much worse for not making the content open and have urls work. Forcing people to register to comment I would be fine with but it is lame that I can’t send the url to people to let them know. LinkedIn has continued to lag behind the social web precisely because of such backwards practices.

    Failing to have urls work kills the social web. Now I guess you can overcome this if you become your own isolated island on the internet with enough users (I guess maybe facebook does this). However, I think it is bad even in those cases. And it is certainly a bad idea in cases where you don’t get the huge user base Facebook does. I suppose LinkedIn feels they can – I don’t think that is wise.

  6. John Hunter
    August 7th, 2010 @ 12:48 pm

    Jose, I agree it is perfectly sensible to bring in ideas of others like Ackoff, Christensen, Drucker… Deming did that all the time. I do see many many failures when organizations try to just skim the surface of management ideas taking a few concepts from here and there. I personally feel you are better off really getting a deep understanding and I think Deming helps this enormously. That said there are many great management organization that don’t explicitly study Deming (though many of these get his ideas, 2nd, 3rd… hand).

  7. Curious Cat Management Improvement Blog » Good Process Improvement Practices
    November 8th, 2010 @ 8:21 am

    one of the aims of the improvement process should be improving peoples ability to improve over the long term (one outcome of the process should be a better process another should be that people learned and can apply what they learned in future improvements)…

  8. Unknown and Unknowable Data « The W. Edwards Deming Institute Blog
    August 29th, 2013 @ 11:15 pm

    […] Related: Manage what you can’t measure – The Idea of Performance Rating to Capture Merit is Alluring – Is the Result Due to Mathematical Probability or Individual Merit? – How to Manage What You Can’t Measure […]

  9. Myth: If You Can’t Measure It, You Can’t Manage It « The W. Edwards Deming Institute Blog
    August 13th, 2015 @ 2:42 pm

    […] wrote a post on my Curious Cat Management Improvement blog about how to manage what you can’t measure (in […]

  10. The Cleveland Clinic Improvement Model, Part 1 | Lean Blog
    January 19th, 2016 @ 7:01 am

    […] Leaders are encouraged to “manage what matters most.” It’s a really popular expression to say “what gets measured, gets managed.” I like the notion of “what matters gets managed.” It’s probably more like “what matters gets measured, so it gets managed.” But. W. Edwards Deming always reminded us that some of the most important things are immeasurable. […]

  11. Undertaking a BPM Initiative and Main Considerations
    September 20th, 2016 @ 11:09 am

    […] Complex situations do not necessarily require a complicated roadmap. If you are tackling a set of processes into which there is little or no visibility, managed by memory, Post-it Notes, Excel spreadsheets, and email, then working on figuring out To Be states may be an exercise which leads to defeat simply because it is such a mess. In such circumstances, your roadmap goal may simply be to bring the Gordian Knot of a mess under control, gaining visibility into what is happening, and thereby bringing shadow processes into the light. Once visibility is achieved, measurement and improvement may then be made – you cannot improve what you cannot see (though John Hunter wrote this interesting piece countering this position on the Curious Cat Manageme…). […]

Leave a Reply





  • Recent Trackbacks

  • Comments