Category Archives: Data

Interview of Bill Hunter: Statistical Variability and Interactions

Interview of Bill Hunter on Statistical Variability and Interactions by Peter Scholtes, 1986:

In this interview Bill Hunter describes how results are made up of the impact and interactions of many variables. Many of those variables we don’t know about or account for. What we normally do is try to figure out the most important variables for processes and then experiment with those variables to find the best options given what we are trying to achieve.

Often the description of what is going on in such cases is that there is arbitrary error or random variation that influences the final results. What Bill discusses in this interview is that what is seen as arbitrary or random is often identifiably caused by specific variables. But often we don’t know what those variables are or how they are varying while we are getting different results over time.

He discusses how many research efforts seek to find the most important 2 variables and create a model based on those 2 variables to predict results. Even in PhD level research that is often done. He then discusses how to deal with other important variables.

He discusses the real world problems businesses must face in creating solutions that work.

If they are going to sell the product in Mississippi, and they are going to sell in Arizona and North Dakota, they have to have a robust product that will work in all these different conditions… It is not good enough for them to have a model that works sometimes… they’ve got to probe deeper and learn how relative humidity affects things and build that into the whole system in a different kind of way… they have to try and dig out the effects of these other x’s

So the business has to figure out the impact of many more variables in order to create reliable and robust products and services. This example is about variables that impact the use of the product by a customer, but the same concept applies to processes within your business.
Continue reading

Interview of Bill Hunter, Brian Joiner and Peter Scholtes on Better Management Practices

Interview with Bill Hunter, Brian Joiner and Peter Scholtes (listen to the interview) on a Public Affair, National Public Radio about improving management practices in the USA. The interview is over 30 years old now but the better management ideas are as true today as they were then.

Sadly we have not improved management practices based on these ideas very much. There have been improvements in how many organizations are managed but those improvements are so slow that fundamental problems remain serious problems decades later.

Brian Joiner

Brian Joiner (quotes from the interview):

You cannot really produce quality in any cost competitive way by relying on inspection to achieve quality. The only way you can really achieve quality in the modern sense is by improving all the processes that go to deliver that product or service. And that requires that you study those processes. And when you study them you very often need to collect and analyze data to find what are the causes of problems.

[We place] a great deal of emphasis on identifying the causes of problems rather than shooting from the hip and jumping to solutions before you really know what the problems are…
Many many dollars, many many hours of time are wasted on “solutions” that are not really solutions.

Bill Hunter:

The problem that employees at Motor Equipment were aware of at the very beginning of this whole business, and for a long time previous – I mean years, was that the city of Madison did not have a preventative maintenance program for vehicles because a mayor said, many years ago said “we fix trucks and other things when they break and then we will save money because we don’t be fiddling with them before.

Well the people out there realized the city was just losing money with this policy so they gathered data, they put it together, they put together a solid case that nobody could argue with that the city should have a preventative maintenance program.

They were able to put together a presentation to the mayor and city council people making their case that there should be a preventative maintenance program. The mayor and the city council people there listened to this presentation of the data and they conclusion was a good proposal.

Continue reading

Factfulness – The Importance of Critical Thinking

Factfulness by Hans Roling (of TED talks and Gapminder charts fame) is an exceptionally good book. It provides great insight into how to think more effectively and how to understand the reality of the world we live in (versus the large distortions so common in most people’s view of the world).

Today the people living in rich countries around the North Atlantic, who represent 11 percent of the world population, make up 60 percent of the Level 4* consumer market. Already by 2027, if incomes keep growing worldwide as they are doing now, then that figure will have shrunk to 50 percent. By 2040, 60 percent of Level 4 consumers will live outside the West.

book cover of Factfulness

One of the significant focuses of the book is the need for critical thinking.

constantly test you favorite ideas for weaknesses. Be humble about the extent of you expertise. Be curious about new information that doesn’t fit, and information from other fields. And rather than talking only to people who agree with you, or collecting examples that fit your ideas, see people who contradict you, disagree with you, and put forward ideas as a great resource for understanding the world.

I have come to see a willingness to value critical thinking, even when it means forcing the organization to address tough issues, as one the differences between organizations that succeed in applying management improvement methods and those that fail. In many organizations that fail, more weight given to making things easy for your bosses versus continual improvement in providing value to customers (which often requires challenging existing processes, beliefs and power structures in the organization).

Challenging the status quo is difficult and most organizations prefer to maintain a culture that takes an easier path. Management improvement often requires a willingness to encourage challenges to the status quo. The importance of challenging the status quo in your organization and in your own thinking is under appreciated.

An example of the systems thinking and economics views Hans shares in the book:

Continue reading

My Willing Worker Award

Here is an image of the employee of the quarter award I received from Bill Scherkenbach.

award certificate - employee of the quarter

I took part in the Deming Red Bead Experiment and earned this award for my exceptional performance.

I have received other awards and I don’t think those awards were given with any more understanding of the contributions to results due to the management systems in those cases than was shown when giving me this award. Even knowing how little impact I could make on the results I was still happy to receive this award: psychology is not always (often? ever?) sensible.

Read some of the lessons from the Red Bead Experiment from my post on The W. Edwards Deming Institute blog:

Data very similar to that provided by the Red Bed Experiment is used everyday in businesses to reward and punish people. Data is used to blame those who fall short of expectations and reward those who have good numbers. In the Red Bead Experiment we know the numbers are not a sensible measure of value provided by the employee. But in our organizations we accept numbers that are just as unrelated to the value provided by the employe to rate and reward employees.

There is a powerful need to improve the numeracy (literacy with numbers) in our organizations. It isn’t a matter of complex math. The concepts are fairly simple…

Related: Guest Post by Bill Scherkenbach – Analysis Must be Implemented by People to Provide ValueRighter Performance AppraisalExperience Teaches Nothing Without Theory

Poorly Stratified Data Leads to Mistakes in Analysis

Getting organization to think of data as critical to making effective decisions is often a challenge. But the very next problem is that while data is used it is actually more misused than used.

How Not to be Wrong (book cover)

What is important is not just having numbers mentioned when decisions are being made. Or even having numbers mentioned when those decisions are evaluated after they have been implemented (or course many organizations don’t even evaluate the results of many changes they adopt, but that is a different problem). What is important for “evidence based decision making” is that what those numbers actually mean must be understood. It is easy to be mislead if you don’t think critically about what the numbers tell you and what they do not.

Poorly stratified data is one problem that leads to mistakes in analysis.

How ZIP codes nearly masked the lead problem in Flint

As I ran the addresses through a precise parcel-level geocoding process and visually inspected individual blood lead levels, I was immediately struck by the disparity in the spatial pattern. It was obvious Flint children had become far more likely than out-county children to experience elevated blood lead when compared to two years prior.

How had the state so blatantly and callously disregarded such information? To me – a geographer trained extensively in geographic information science, or computer mapping – the answer was obvious upon hearing their unit of analysis: the ZIP code.

Their ZIP code data included people who appeared to live in Flint and receive Flint water but actually didn’t, making the data much less accurate than it appeared [emphasis added].

This type of assumption about data leading to mistakes in analysis is common. The act of using data doesn’t provide benefits is the data isn’t used properly. The more I see of the misuse of data to more importance I place on the skill of thinking critically. We must challenge assumptions and challenge what the data we look at actually means.

Continue reading

Don’t Expect Short Quotes to Tell the Whole Story

When people try to use a short quote as an accurate encapsulation of a management concept they will often be disappointed.

It is obvious that Dr. Deming believed that organizations failed to use data effectively to improve needed to change and use data effectively in order to thrive over the long term. He believed that greatly increasing the use of data in decision making would be useful. He also believe there were specific problems with how data was used, when it is was used. Failing to understand variation leads to misinterpreting what conclusions can appropriately be drawn from data.

Using data is extremely useful in improving performance. But as Deming quoted Lloyd Nelson as saying “the most important figures that one needs for management are unknown or unknowable.”

I believe Dr. Deming would have said something like “In God we trust, all others bring data” (I haven’t been able to find a source verifying he did say it). Others don’t believe he would referencing the Lloyd Nelson quote and all Deming’s other work showing that Dr. Deming’s opinion that data isn’t all that matters. I believe they are correct that Dr. Deming wouldn’t mean for the quote to be taken literally as a summation of everything he ever said. That doesn’t mean he wouldn’t use a funny line that emphasized an important message – we need to stop relying so much on unsubstantiated opinion and instead back up opinion with data (including experiments).

Quotes can help crystallize a concept and drive home a point. They are very rarely a decent way to pass on the whole of what the author meant, this is why context is so important. But, most often quotes are shared without context and that of course, leads to misunderstandings.

image of quote - It is wrong to suppose that if you can't measure it, you can't manage it - a costly myth.

A funny example of this is the Deming quote that you often see: “if you can’t measure it, you can’t manage it.” Deming did actually say that. But without the context you get 100% the wrong understanding of what he said. Deming’s full statement is “It is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth.” Now normally much more context is required to truly understand the author’s point. But this is a funny example of how a quote can be even be accurate when passed on to you and yet completely misleading because it is taken out of context.

Continue reading

The Importance of Critical Thinking and Challenging Assumptions

There are many factors that are important to effectively practice the management improvement ideas I have discussed in this blog for over a decade. One of the most important is a culture that encourages critical thinking as well as challenging claims, decisions and assumptions.

I discussed this idea some in: Customers Are Often Irrational. There is a difference between saying people wish to have their desires met and people act in the manner to maximize the benefits they wish to receive.

It is important to study choices customers make and learn from them. But being deceived by what their choices mean is easier than is usually appreciated. Often the decision made is contrary to the ideal choice based on their beliefs. It is often poor decision making not an indication that really they want a different result than they express (as revealed versus stated preference can show). People that ignore the evidence behind climate change and condemn coastal areas to severe consequences don’t necessarily prefer the consequences that their decision leads to. It may well be that decision to ignore the evidence is not based on a desire to suffer long term consequences in order to get short term benefits. It may well be just an inability to evaluate evidence in an effective way (fear of challenging ourselves to learn about matters we find difficult often provides a strong incentive to avoid doing so).

Knowing the difference between choosing short term benefits over long term consequences and a failure to comprehend the long term consequences is important. Just as in this example, many business decisions have at the root a desire to pretend we can ignore the consequences of our decisions and a desire to accept falsehoods that let us avoid trying to cope with the difficult problems.

photo of me with a blackboard in my father's office

Photo of me and my artwork in my father’s office by Bill Hunter

It is important to clearly articulate the details of the decision making process. We need to note the actual criticism (faulty logic, incorrect beliefs/assumptions…) that results in what some feel is a poor conclusion. But we seem to find shy away from questioning faulty claims (beliefs that are factually incorrect – that vaccines don’t save people from harm, for example) or lack of evidence (no data) or poor reasoning (drawing unsupported conclusions from a well defined set of facts).

Critical thinking is important to applying management improvement methods effectively. It is important to know when decisions are based on evidence and when decisions are not based on evidence. It can be fine to base some decisions on principles that are not subject to rational criticism. But it is important to understand the thought process that is taken to make each decision. If we are not clear on the basis (evidence or opinion regardless of evidence) we cannot be as effective in targeting our efforts to evaluate the results and continually improve the processes in our organizations.

Describing the decision as “irrational” is so imprecise that it isn’t easy to evaluate how much merit the criticism has. If specific facts are called into question or logical fallacies within the decision making process are explained it is much more effective at providing specific items to explore to evaluate whether the criticism has merit.

When specific criticisms are made clear then those supporting such a decision can respond to the specific issues raised. And in cases where the merits of one course of action cannot be agreed to then such critical thought can often be used to create measures to be used to evaluate the effectiveness of the decision based on the results. Far too often the results are not examined to determine if they actually achieved what was intended. And even less often is care taken to examine the unintended consequences of the actions that were taken.

Continue reading

Technological Innovation and Management

Technological innovation brings great opportunity for improving results and our quality of life. But transforming potential benefits into real results comes with many challenges.

ASQ has asked their Influential Voices to explore the idea of the fourth industrial revolution: “this new era is founded on the practical use of technological innovations like artificial intelligence, big data, robotics, and the Internet of Things (IoT).”

For many years GMs huge investment in robotics in the 1980s ($billions) has been an example of how pinning hopes on technology often doesn’t produce the desired results. I think that a capable management system is needed to make technological innovation as successful as it needs to be.

In this decade we are finally reaching the point where robotics is really making incredible strides. Robotics has provided huge benefits for decades, when used appropriately, but the ease of use and benefits from robotics have greatly increased recently.

I think robotics is going to be an incredibly powerful source of benefits to society in the next 20 years. Amazon is very well placed to profit in this area. Several other companies (Toyota, Boston Dynamics*, Honda, SoftBank…) are likely to join them (though which will be the biggest winners and which will stumble is not obvious)

Cliff Palace historical ruins

Photo by John Hunter of Cliff Palace (built in the 1190s), Mesa Verde National Park.

I am less confident in the Internet of Things. It seems to me that much of the IoT effort currently is flailing around in ways similar to GMs approach to robotics in the 1980s and 1990s. There is huge potential for IoT but the architecture of those solutions and the impact of that architecture on security (and fragile software that creates many more problems than it solves) is not being approached wisely in my opinion. IoT efforts should focus on delivering robust solutions in the areas where there is a clear benefit to adopting IoT solutions. And that needs to be done with an understanding of security and the lifecycle of the devices and businesses.

I think it will be much wiser to have an internet hub in the business or home that has all IoT traffic route through it in a very clear and visible way. Users need clear ways to know what the IoT is trying to do and to have control to determine what is and what is not sent out from their system. Having devices that share information in a non-transparent way is not wise. This is especially when those devices have cameras or microphones.

Continue reading

All Data is Wrong, Some is Useful

From my first blog post on this blog – Dangers of Forgetting the Proxy Nature of Data

we often fail to explore whether changes in the numbers (which we call results) are representative of the “true results” of the system or if the data is misleading.

Data is meant to provide us insight into a more complex reality. We need to understand the limitations when we look at “results” and understand data isn’t really the results but a representation we hope is close to reality so we can successfully use the data to make decisions.

But we need to apply thought to how we use data. Lab results are not the same are what happens in the field. It is cheaper and faster to examine results in a lab. But relying on lab results involves risk. That doesn’t mean relying on lab results is bad, we have to balance the costs and benefits of getting more accurate data.

But relying on lab results and not understanding the risk is dangerous. This is the same idea of going to the gemba to get an accurate understanding instead of relying on your ability to imagine reality based upon some data and ideas of what it is probably like.

photo of a Modified Yellow VW Beetle

VW Beetle (in Bangkok, Thailand) has some sort of modification along the back bumper but I don’t know what it is meant to do. Any ideas? More of my photos from Bangkok.

Volkswagen Drops 23% After Admitting Diesel Emissions Cheat

Volkswagen AG lost almost a quarter of its market value after it admitted to cheating on U.S. air pollution tests for years

During normal driving, the cars with the software — known as a “defeat device” — would pollute 10 times to 40 times the legal limits, the EPA estimated. The discrepancy emerged after the International Council on Clean Transportation commissioned real-world emissions tests of diesel vehicles including a Jetta and Passat, then compared them to lab results.

Obviously VW was managing-to-test-result instead of real world value. It seems they were doing so intentionally to provide misleading data. Obviously one of the risks with lab test results (medical trials etc.) is that those with an interest in showing better results could manipulate the data and lab procedures (or systems) to have the data show their product in the most favorable light.

Continue reading

Transforming a Management System – A Case Study From the Madison Wisconsin Police Department

This post in an excerpt from The Quality Leadership Workbook for Police by Chief David Couper and Captain Sabine Lobitz (buy via Amazon).

cover image of the New Quality Leadership Workbook for Police

The New Quality Leadership Workbook for Police

Transformational Steps
A Case Study Madison, Wisconsin (1981-1993)

Step 1: Educate and inform everyone in the organization about the vision, the goals, and Quality Leadership. This step must be passionately led by the top leader.

  • Begin discussion with top management team and train them.
  • Discuss and ask employees; get feedback from them.
  • Share feedback with the chief and his management team.
  • Get buy-in from top department managers.
  • Survey external customers—citizens; those who live and work in the community.
  • Create an employee’s advisory council; ask, listen, inform, and keep them up to date on what’s going on.
  • The chief keeps on message; tells, sells, and persuades, newsletters, meetings and all available media.

Step 2: Prepare for the transformation. Before police services to the community can be improved, it is essential to prepare the inside first — to cast a bold vision and to have leaders that would “walk the talk.”

  • Appoint a top-level, full-time coordinator to train, coach, and assist in the transformation.
  • Form another employee council to work through problems and barriers encountered during implementation of the transformation and Quality Leadership.
  • Require anyone who seeks to be a leader to have the knowledge and ability to practice Quality Leadership.

Step 3: Teach Quality Leadership. This begins at the top with the chief and the chief’s management team.

  • Train all organizational leaders in Quality Leadership.
  • Train all employees as to what Quality Leadership is, why the transformation is necessary, and what it means for them.

Step 4: Start practicing Quality Leadership. If top managers within the organization are not authentically practicing Quality Leadership neither will anyone else.

Continue reading