Currently browsing the Data Category

posts relating to data, data based decisions, statistics, SPC, data quality, data analysis, data collection... It is critical to understand common and special cause variation.
Recommended posts: Data is Only a Proxy - Targets Distorting the System - Measurement and Data Collection - Evidence-based Management - Visible Data - Understanding Data
Related: Operational definition - Control chart


Out of Touch Executives Damage Companies: Go to the Gemba

When your customer service organization is universally recognized as horrible adding sales requirements to customer service representatives jobs is a really bad practice. Sadly it isn’t at all surprising to learn of management doing just that at our largest companies. Within a system where cash and corruption buys freedom from market forces (see below for more details) such practices can continue.

Such customer hostile practices shouldn’t continue. They shouldn’t be allowed to continue. And even though the company’s cash has bought politically corrupt parties to allow such a system to survive it isn’t even in the selfish interest of the business. They could use the cover provided by bought-and-paid-for-politicians-and-parties to maintain monopolistic pricing (which is wrong ethically and economically but could be seen as in the self interest of a business). But still provide good service (even while you take monopolistic profits allowed with corrupt, though legal, cash payments).

Of course, Adam Smith knew the likely path to corruption of markets made up of people; and he specifically cautioned that a capitalist economic system has to prevent powerful entities efforts to distort markets for individual gain (perfect competition = capitalism, non-competitive markets = what business want, as Adam Smith well knew, but this is precisely not capitalism). Sadly few people taking about the free-market or capitalism understand that their support of cronyist policies are not capitalist (I suppose some people mouthing those words are just preaching false ideas to people known to be idiots, but really most don’t seem to understand capitalism).

Anyway, this class of protected businesses supported by a corrupt political and government (regulators in government) sector is a significant part of the system that allows the customer hostility of those politically connected large businesses to get away with a business model based on customer hostility, but wasn’t really what I meant to write about here.

Comcast executives have to know they are running a company either rated the worst company in the country or close to it year after year. They, along with several others in their industry, as well as the cell phone service providers and too-big-to-fail-banks routinely are the leaders of companies most reviled by customers. Airlines are also up their for treating customer horribly but they are a bit different than the others (political corruption is much less of the reason for their ability to abuse customers for decades than is for the others listed above).

Leaked Comcast employee metrics show what we figured: Sell or perish [Updated]
Training materials explicitly require a “sell” phase, even in support calls.

The company’s choice to transform what is traditionally a non-revenue-generating area—customer service—into a revenue-generating one is playing out with almost hilariously Kafkaesque consequences. It is the nature of large corporations like Comcast to have dozens of layers of management through which leadership instructions and directives are filtered. The bigger the company, the more likely that members of senior leadership (like Tom Karinshak) typically make broad policy and leave specific implementations to lower levels.

Here, what was likely praised in the boardroom as an “innovative” strategy to raise revenue is instead doing much to alienate customers and employees alike. Karinshak’s assurances that he doesn’t want employees to feel pressured to sell in spite of hard evidence that Comcast demands just that are hard to square with the content of the document.

So what is going on here? Most people can easily see this is likely a horrible practice. It is a practice that a well run company theoretically could pull off without harming customers too much. But for a company like Comcast to do this it is obviously going to be horrible for customers (same for all those too-big to fail banks, cell phone service providers and other ISPs and cable TV providers).

Lets just pretend Comcast’s current leadership executives were all replaced with readers of the Curious Cat Management Improvement blog. And lets say that for now you are suppose to focus on improving the policies in place (while thinking about policy changes for later but not making them yet).

Continue reading

George Box Webcast on Statistical Design in Quality Improvement

George Box lecture on Statistical Design in Quality Improvement at the Second International Tampere Conference in Statistics, University of Tampere, Finland (1987).

Early on he shows a graph showing the problems with American cars steady over a 10 years period. Then he overlays the results for Japanese cars which show a steady and significant decline of the same period.

Those who didn’t get to see presentations before power point also get a chance to see old school, hand drawn, overhead slides.

He discusses how to improve the pace of improvement. To start with informative events (events we can learn from) have to be brought to the attention of informed observers. Otherwise only when those events happen to catch the attention of the right observer will we capture knowledge we can use to improve. This results in slow improvement.

A control chart is an example of highlighting that something worth studying happened. The chart will indicate when to pay attention. And we can then improve the pace of improvement.

Next we want to encourage directed experimentation. We intentionally induce informative events and pay close attention while doing so in order to learn.

Every process generates information that can be used to improve it.

He emphasis the point that this isn’t about only manufacturing but it true of any process (drafting, invoicing, computer service, checking into a hospital, booking an airline ticket etc.).

He then discussed an example from a class my father taught and where the students all when to a TV plant outside Chicago to visit. The plant had been run by Motorola. It was sold to a Japanese company that found there was a 146% defect rate (which meant most TVs were taken off the line to be fixed at least once and many twice) – this is just the defect rate before then even get off the line. After 5 years the same plant, with the same American workers but a Japanese management system had reduced the defect rate to 2%. Everyone, including managers, were from the USA they were just using quality improvement methods. We may forget now, but one of the many objections managers gave for why quality improvement wouldn’t work in their company was due to their bad workers (it might work in Japan but not here).

He references how Deming’s 14 points will get management to allow quality improvement to be done by the workforce. Because without management support quality improvement processes can’t be used.

With experimentation we are looking to find clues for what to experiment with next. Experimentation is an iterative process. This is very much the mindset of fast iteration and minimal viable product (say minimal viable experimentation as voiced in 1987).

There is great value in creating iterative processes with fast feedback to those attempting to design and improve. Box and Deming (with rapid turns of the PDSA cycle) and others promoted this 20, 30 and 40 years ago and now we get the same ideas tweaked for startups. The lean startup stuff is as closely related to Box’s ideas of experimentation as an iterative process as it is to anything else.

Related: Ishikawa’s seven quality control tools

He also provided a bit of history that I was not aware of saying the first application of orthogonal arrays (fractional factorial designs) in industry was by Tippett in 1933. And he then mentioned work by Finney in 1945, Plackett and Burman in 1946 and Rao in 1947.

Taking Risks Based on Evidence

My opinion has long been that football teams are too scared to take an action that is smart but opens the coach to criticism. So instead of attempting to make it on 4th down (if you don’t understand American football, just skip this post) they punt because that is the decision that is accepted as reasonable.

So instead of doing what is wise they do what avoids criticism. Fear drives them to take the less advantageous action. Now I have never looked hard at the numbers, but my impression is that it is well worth the risk to go for it on 4th down often. In a quick search I don’t see a paper by a Harvard professor (this article refers to it also – Fourth down: To punt or to go?) on going for it on 4th down but I found on by a University of California, Berkeley economist (David Romer wrote called “Do Firms Maximize? Evidence from Professional Football.”).

On the 1,604 fourth downs in the sample for which the analysis implies that teams are on average better off kicking, they went for it only nine times. But on the 1,068 fourth downs for which the analysis implies that teams are on average better off going for it, they kicked 959 times.

My guess is that the advantages to going for it on 4th down are greater for high school than college which is greater than the advantage for the pros (but I may be wrong). My guess is this difference is greater the more yardage is needed. Basically my feeling is the variation in high school is very high in high school and decreases with greater skill, experience and preparation. Also the kicking ability (punting and field goals) impacts the choices of going for it on 4th down and that dramatically increases in college. So if I am correct, I think pro coaches should be more aggressive on 4th down, but likely less aggressive than high school coaches should be.

But in any event the data should be explored and strategies should be tested.

Continue reading

Stu Hunter Discussing Bill Hunter, Statistics for Experimenters and EVOP

In this clip, Stu Hunter talks about Bill Hunter (my father, and no relation to Stu Hunter), Statistics for Experimenters and EVolutionary OPerations (EVOP).

Stu mentions Bill Hunter’s work with the City of Madison, which started with the First Street Garage (Out of the Crisis included a short write up on this effort by Dad, which, I believe, was the first application of Deming’s ideas in the public sector).

There was also a great deal of work done with the Police department, as the police chief, David Couper, saw great value in Deming’s ideas. The Police department did some great work and David’s blog shares wonderful ideas on improving policing. I don’t think Dad was that directly involved in what happened there, but it is one of the nice benefits of seeding new ideas: as they take root and grow wonderful things happen without any effort on your part.

As to why Dad got involved with the city, he returned from a summer teaching design of experiments and quality improvement methods in China (this is just before China was really open, a few outsiders were let in to teach). We had also lived overseas several other times, always returning to Madison. He decided he wanted to contribute to the city he loved, Madison, and so he talked to the Mayor about helping improve performance of the city.

The mayor listened and they started with a pilot project which Dad work on with Peter Scholtes. Dad talked to Peter, who he had know for years, and who worked for the city, before talking to the mayor. Read more about the efforts in Madison via the links at the end of this post.

Continue reading

The Art of Discovery

Quality and The Art of Discovery by Professor George Box (1990):


Quotes by George Box in the video:

“I think of statistical methods as the use of science to make sense of numbers”

“The scientific method is how we increase the rate at which we find things out.”

“I think the quality revolution is nothing more, or less, than the dramatic expansion of the of scientific problem solving using informed observation and directed experimentation to find out more about the process, the product and the customer.”

“It really amounts to this, if you know more about what it is you are doing then you can do it better and you can do it cheaper.”

“We are talking about involving the whole workforce in the use of the scientific method and retraining our engineers and scientists in a more efficient way to run experiments.”

“Tapping into resources:

  1. Every operating system generates information that can be used to improve it.
  2. Everyone has creativity.
  3. Designed experiments can greatly increase the efficiency of experimentation.

An informed observer and directed experimentation are necessary for the scientific method to be applied. He notes that the control chart is used to notify an informed observer to explain what is special about the conditions when a result falls outside the control limits. When the chart indicates a special cause is likely present (something not part of the normal system) an informed observer should think about what special cause could lead to the result that was measured. And it is important this is done quickly as the ability of the knowledgable observer to determine what is special is much greater the closer in time to the result was created.

The video was posted by Wiley (with the permission of George’s family), Wiley is the publisher of George’s recent autobiography, An Accidental Statistician: The Life and Memories of George E. P. Box, and many of his other books.

Related: Two resources, largely untapped in American organizations, are potential information and employee creativityStatistics for Experimenters (book on directed experimentation by Box, Hunter and Hunter)Highlights from 2009 George Box SpeechIntroductory Videos on Using Design of Experiments to Improve Results (with Stu Hunter)

George Box

I would most likely not exist if it were not for George Box. My father took a course from George while my father was a student at Princeton. George agreed to start the Statistics Department at the University of Wisconsin – Madison, and my father followed him to Madison, to be the first PhD student. Dad graduated, and the next year was a professor there, where he and George remained for the rest of their careers.

George died today, he was born in 1919. He recently completed An Accidental Statistician: The Life and Memories of George E. P. Box which is an excellent book that captures his great ability to tell stories. It is a wonderful read for anyone interested in statistics and management improvement or just great stories of an interesting life.

photo of George EP Box

George Box by Brent Nicastro.

George Box was a fantastic statistician. I am not the person to judge, but from what I have read one of the handful of most important applied statisticians of the last 100 years. His contributions are enormous. Several well know statistical methods are known by his name, including:

George was elected a member of the American Academy of Arts and Sciences in 1974 and a Fellow of the Royal Society in 1979. He also served as president of the American Statistics Association in 1978. George is also an honorary member of ASQ.

George was a very kind, caring and fun person. He was a gifted storyteller and writer. He had the ability to present ideas so they were easy to comprehend and appreciate. While his writing was great, seeing him in person added so much more. Growing up I was able to enjoy his stories often, at our house or his. The last time I was in Madison, my brother and I visited with him and again listened to his marvelous stories about Carl Pearson, Ronald Fisher and so much more. He was one those special people that made you very happy whenever you were near him.

George Box, Stuart Hunter and Bill Hunter (my father) wrote what has become a classic text for experimenters in scientific and business circles, Statistics for Experimenters. I am biased but I think this is acknowledged as one of (if not the) most important books on design of experiments.

George also wrote other classic books: Time series analysis: Forecasting and control (1979, with Gwilym Jenkins) and Bayesian inference in statistical analysis. (1973, with George C. Tiao).

George Box and Bill Hunter co-founded the Center for Quality and Productivity Improvement at the University of Wisconsin-Madison in 1984. The Center develops, advances and communicates quality improvement methods and ideas.

The Box Medal for Outstanding Contributions to Industrial Statistics recognizes development and the application of statistical methods in European business and industry in his honor.

All models are wrong but some are useful” is likely his most famous quote. More quotes By George Box

A few selected articles and reports by George Box

Continue reading

Quality Processes in Unexpected Places

This month Paul Borawski asked ASQ’s Influential Voices to explore the use of quality tools in unexpected places.

The most surprising example of this practice that I recall is the Madison, Wisconsin police department surveying those they arrested to get customer feedback. It is obvious that such “customers” are going to be biased. Still the police department was able to get actionable information by seeking the voice of the customer.

photo of a red berry and leaves

Unrelated photo from Singapore Botanical Garden by John Hunter.

Certain of the police department’s aims are not going to match well with those they arrest (most obviously those arrested wish the police department didn’t arrest them). The police department sought the voice of the customer from all those they interacted with (which included those they arrested, but also included those reporting crimes, victims, relatives of those they arrested etc.).

The aim of the police department is not to arrest people. Doing so is necessary but doing so is most similar in the management context to catching an error to remove that bad result. It is better to improve processes so bad results are avoided. How the police interact with the public can improve the process to help steer people’s actions away from those that will require arrests.

The interaction police officers have with the public is a critical gemba for meeting the police department’s aim. Reducing crime and encouraging a peaceful society is aided by knowing the conditions of that gemba and knowing how attempts to improve are being felt at the gemba.

All customer feedback includes bias and personal preferences and potentially desires that are contrary to the aims for the organization (wanting services for free, for example). Understanding this and how important understanding customer/user feedback on the gemba is, it really shouldn’t be surprising that the police would want that data. But I think it may well be that process thinking, evidence based management and such ideas are still not widely practiced as so the Madison police department’s actions are still surprising to many.

Quality Leadership: The First Step Towards Quality Policing by David Couper and Sabine Lobitz

Our business is policing, our customers are the citizens within our jurisdictions, and our product is police service (everything from crime fighting and conflict management to safety and prevention programs.)

If we are to cure this we must start to pay attention to the new ideas and trends in the workplace mentioned earlier that are helping America’s businesses; a commitment to people, how people are treated — employees as well as citizens, the development of a people-oriented workplace, and leadership can and does make a difference.

If we change the way in which we lead the men and women in our police organizations, we can achieve quality in policing. However, wanting to change and changing are worlds apart. The road to change is littered by good intentions and short-term efforts.

This article, from 1987, illustrates the respect for people principle was alive and being practiced 25 years ago; most organizations need to do a great deal more work on applying practices that show respect for people.

Related: Quality Improvement and Government: Ten Hard Lessons From the Madison Experience by David C. Couper, Chief of Police, City of Madison, Wisconsin – SWAT Raids, Failure to Apply System Thinking in Law EnforcementMeasuring What Matters: Developing Measures of What the Police DoThe Public Sector and W. Edwards DemingDoing More with Less in the Public Sector – A Progress Report from Madison, Wisconsin

The Market Discounts Proven Company Leadership Far Too Quickly

Developing a strong executive leadership culture is not a short term effort. It isn’t based on one person. It almost never deteriorates quickly. Yet markets continually overact to minor blips on the long term success of companies. I think this is mainly due to a failure to appreciate systems and a failure to appreciate variation along with plenty of other contributing factors.

The market’s weakness does provide investment opportunities. Though taking advantages of them is much more difficult than spotting a general weakness. While excellent management almost never becomes pitiful overnight (regardless of how often talking heads would have you believe) business can change very quickly due to rapidly changing market conditions. Avoiding the purchases when the underlying business has sustained a significant blow that excellent management will deal with but which will reduce the value of the enterprise going forward is key to taking advantage of the market’s silly overreaction to bad news (or even calling things “bad news” that are not actually bad just not as awesome as some were hoping for).

My positive opinion of Toyota’s management has continued for a long time. A few years ago an amazing number of people were all excited about the “decline of Toyota” and wrote about how Toyota’s ways had to change. I wrote at the time was this is needless hysteria and if Toyota just focused a bit more on applying the Toyota’s management methods they would be in great shape. The problems were due to Toyota’s mistakes in practicing the Toyota Production System not in a weakness of those practices.

Looking at a chart of Toyota’s stock price from 2007 to today it peaks at about $137 in January 2007 and bottoms at $58 in early 2009 and now is at $96. Toyota’s stock price has been priced richly due to respect for management and consistently strong cash flow. As it fell below $75 there you no longer had to pay a premium for excellent management, but that management was still there. I like getting bargains when I buy stocks. One of the things I have learned I am too focused on bargains and I should be more willing to accept less of a bargain to get great management systems – so I have adjusted, and have improved my results. When I can get a great bargain and great management it is wonderful, though sadly a rare occurrence. Toyota’s price now seems reasonable, but not a huge bargain.

The market continually gets overly excited by either actual problems or perceived problems. I wrote about this happening with Netflix 2 years ago. Netflix made some mistakes and faced some tough business issues. The evidence of sound, sensible, effective management vastly outweighed the evidence for management failure – yet there were hundreds of articles about the pitiful failure of Netflix management.

Continue reading

Leanpub Podcast on My Book – Management Matters: Building Enterprise Capability

image of the cover of Managmenet Matters by John Hunter

Management Matters by John Hunter

I recently was interviewed for a podcast by Len Epp with Leanpub: Leanpub Podcast Interview #9: John Hunter. I hope you enjoy the podcast (download the mp3 of the podcast).

In the podcast we cover quite a bit of ground quickly, so the details are limited (transcript of the interview). These links provide more details on items I mention in the podcast. They are listed below in the same order as they are raised in the podcast:

The last 15 minutes of the podcast I talk about some details of working with Leanpub; I used Leanpub to publish Management Matters. I recommend Leanpub for other authors. They don’t just have lean in their name, they actual apply lean principles (focusing on the value chain, eliminating complexity, customer focus, etc.) to operating Leanpub. It is extremely easy to get started and publish your book.

Leanpub also offers an excellent royalty plan: authors take home 90% of the revenue minus 50 cents per book. They publish without “digital rights management” crippling purchasers use of the books. Buyers have access to pdf, kindle (mobi) and epub (iPad, nook) format books and get access to all updates to the book. All purchases include a 45 day full money back guaranty.

Related: Business 901 Podcast with John Hunter: Deming’s Management Ideas TodayInterviews for Management Matters: Building Enterprise Capability

Indirect Improvement

Often the improvements that have the largest impact are focused on improving the effectiveness of thought and decision making. Improving the critical thinking in an organization has huge benefits over the long term.

My strategy along the lines of improving critical thinking is not to make that the focus of some new effort. Instead that ability to reason more effectively will be an outcome of things such as: PDSA projects (where people learn that theories must be tested, “solutions” often fail if you bother to look at the results…), understanding variation (using control charts, reading a bit of material on: variation, using data effectively, correlation isn’t causation etc.), using evidenced based management (don’t make decision based on the authority of the person speaking but on the merit that are spoken).

These things often take time. And they support each other. As people start to understand variation the silly discussion of what special causes created the result that is within the expected outcomes for the existing process are eliminated. As people learn what conclusions can, and can’t, be drawn from data the discussions change. The improvements from the process of making decisions is huge.

As people develop a culture of evidence based management if HiPPOs try to push through decision based on authority (based on Highest Paid Person’s Opinion) without supporting evidence those attempts are seen for what they are. This presents a choice where the organization either discourages those starting to practice evidence based decision making (reverting to old school authority based decision making) or the culture strengthens that practice and HiPPO decision making decreases.

Building the critical thinking practices in the organization creates an environment that supports the principles and practices of management improvement. The way to build those critical thinking skills is through the use of quality tools and practices with reminders on principles as projects are being done (so until understanding variation is universal, continually pointing out that general principle with the specific data in the current project).

The gains made through the direct application of the tools and practices are wonderful. But the indirect benefit of the improvement in critical thinking is larger.

Related: Dan’t Can’t LieGrowing the Application of Management Improvement Ideas in Your OrganizationBuild Systems That Allow Quick Action – Don’t Just Try and Run FasterBad Decisions Flow From Failing to Understand Data and Failing to Measure Results of Changes

Special Cause Signal Isn’t Proof A Special Cause Exists

One of my pet peeves is when people say that a point outside the control limits is a special cause. It is not. It is an indication that it likely a special cause exists, and that special cause thinking is the correct strategy to use to seek improvement. But that doesn’t mean there definitely was a special cause – it could be a false signal.

This post relies on an understand of control charts and common and special causes (review these links if you need some additional context).

Similarly, a result that doesn’t signal a special cause (inside the control limits without raising some other flag, say a run of continually increasing points) does not mean a special cause is not present.

The reason control charts are useful is to help us maximize our effectiveness. We are biased toward using special cause thinking when it is not the most effective approach. So the control chart is a good way to keep us focused on common cause thinking for improvement. It is also very useful in flagging when it is time to immediately start using special cause thinking (since timing is key to effective special cause thinking).

However, if there is result that is close to the control limit (but inside – so no special cause is indicated) and the person that works on the process everyday thinks, I noticed x (some special cause) earlier, they should not just ignore that. It very well could be a special cause that, because of other common cause variation, resulted in a data point that didn’t quite reach the special cause signal. Where the dot happened to land (just above or just below the control limit – does not determine if a special cause existed).

The signal is just to help us systemically make the best choice of common cause or special cause thinking. The signal does not define whether a special cause (an assignable cause) exists of not. The control chart tool helps guide us to use the correct type of improvement strategy (common cause or special cause). But it is just a signaling device, it isn’t some arbiter of whether a special cause actually exists.

Continue reading

Introductory Videos on Using Design of Experiments to Improve Results

The video shows Stu Hunter discussing design of experiments in 1966. It might be a bit slow going at first but the full set of videos really does give you a quick overview of the many important aspects of design of experiments including factorial designed experiments, fractional factorial design, blocking and response surface design. It really is quite good, if you find the start too slow for you skip down to the second video and watch it.

My guess is, for those unfamiliar with even the most cursory understanding of design of experiments, the discussion may start moving faster than you can absorb the information. One of the great things about video is you can just pause and give yourself a chance to catch up or repeat a part that you didn’t quite understand. You can also take a look at articles on design of experiments.

I believe design of experiments is an extremely powerful methodology of improvement that is greatly underutilized. Six sigma is the only management improvement program that emphasizes factorial designed experiments.

Related: One factor at a time (OFAT) Versus Factorial DesignsThe purpose of Factorial Designed Experiments

Continue reading

Keys to the Effective Use of the PDSA Improvement Cycle

The PDSA improvement cycle was created by Walter Shewhart where Dr. Deming learned about it. An improvement process is now part of many management improvement methods (A3 for lean manufacturing, DMAIC for six sigma and many other modifications). They are fairly similar in many ways. The PDSA cycle (Plan, Do, Study, Act) has a few key pieces that are either absent in most others processes of greatly de-emphasized which is why I prefer it (A3 is my second favorite).

The PDSA cycle is a learning cycle based on experiments. When using the PDSA cycle prediction of the results are important. This is important for several reasons but most notably due to an understanding of the theory of knowledge. We will learn much more if we write down our prediction. Otherwise we often just think (after the fact); yeah that is pretty much what I expected (even if it wasn’t). Also we often fail to think specifically enough at the start to even have a prediction. Forcing yourself to make a prediction gets you to think more carefully up front and can help you set better experiments.

An organization using PDSA well will turn the PDSA cycle several times on any topic and do so quickly. In a 3 month period turning it 5 times might be good. Often those organizations that struggle will only turn it once (if they are lucky and even reach the study stage). The biggest reason for effective PDSA cycles taking a bit longer is wanting more data than 2 weeks provides. Still it is better to turn it several times will less data – allowing yourself to learn and adjust than taking one long turn.

The plan stage may well take 80% (or even more) of the effort on the first turn of the PDSA cycle in a new series. The Do stage may well take 80% of of the time – it usually doesn’t take much effort (to just collect a bit of extra data) but it may take time for that data to be ready to collect. In the 2nd, 3rd… turns of the PDSA cycle the Plan stage often takes very little time. Basically you are just adjusting a bit from the first time and then moving forward to gather more data. Occasionally you may learn you missed some very important ideas up front; then the plan stage may again take some time (normally if you radically change your plans).

Remember to think of Do as doing-the-experiment. If you are “doing” a bunch of work (not running an experiment and collecting data) that probably isn’t “do” in the PDSA sense.

Study should not take much time. The plan should have already have laid out what data is important and an expectation of what results will be achieved and provide a good idea on next steps. Only if you are surprised (or in the not very common case that you really have no idea what should come next until you experiment) will the study phase take long.

Continue reading

Agile Story Point Estimation

In agile software development tasks are documented as user stories. Then the level of effort for those stores can be estimated by assigning each story points. The velocity that can be produced in a period (called a sprint, for us 2 weeks) can be estimated. Thus you can predict what can be delivered in the next sprint (which can help business managers make priority decisions).

I have found estimation to be worthwhile. In doing so, we accept there is a great amount of variation but points give a hint to scale. They can help prioritize (if you have 5 things you want but 1 is much harder you may well drop that to the bottom). I have always accepted a great amount of variation in the velocity, worry about the variation I don’t find worthwhile. I do think trying to act as though the velocity is precise can lead to problems. At the same time having a measure of velocity, even accepting understanding variation was present, was useful.

Over time reducing variation (probably largely through better estimation and perhaps a few better tools, reduced technical debt, better documentation, testing…) is helpful and laudable. We made improvement but still lots of variation existed. The biggest help in reducing the measured velocity was breaking down large stories to more manageable sizes. The challenge of estimating user stories, I suspect, has some fairly high variation (even with good system improvements that can help reduce variation).

Large stories just can hide huge variation in what is really required once getting into implementing it.

The way we did estimation (discussing in a sprint planning meeting) did take some time (but not a huge amount). It was agreed by those involved that the time spent was worthwhile. Sometimes we did slip and spend too much time on this, that was an area we had to pay attention to. The discussions were educational and helped provide guidance on how to approach the story. The value of discussions around estimations was probably the biggest surprise I have had in implementing any agile ideas. The value of those discussion was much higher than I imagined (I basically anticipated them just as non-value added time to get the result of an estimate, but they were a source of learning and consensus building).

Related: Assigning Story Points to Bug FixesMistake Proofing the Deployment of Software CodeChecklists in Software Development

These thoughts were prompted by: Story Points Considered Harmful – Or why the future of estimation is really in our past…

Continue reading

Trust But Verify

The following are my comments, which were sparked by question “Trust, but verify. Is this a good example of Profound Knowledge in action?” on the Linked In Deming Institute group.

Trust but verify makes sense to me. I think of verify as process measures to verify the process is producing as it should. By verifying you know when the process is failing and when to look for special causes (when using control chart thinking with an understanding of variation). There are many ways to verify that would be bad. But the idea of trust (respect for people) is not just a feel-good, “be nice to everyone and good things happen”, in Deming’s System of Profound Knowledge.

I see the PDSA improvement cycle as another example of a trust-but-verify idea. You trust the people at the gemba to do the improvement. They predict what will happen. But they verify what does actually happen before they run off standardizing and implementing. I think many of us have seen what happens when the idea of letting those who do the work, improve the process, is adopted without a sensible support system (PDSA, training, systems thinking…). It may actually be better than what was in place, but it isn’t consistent with Deming’s management system to just trust the people without providing methods to improve (and education to help people be most effective). Systems must be in place to provide the best opportunity to succeed. Trusting the people that do the work, is part of it.

I understand there are ways to verify that would be destructive. But I do believe you need process measures to verify systems are working. Just trusting people to do the right thing isn’t wise.

A checklist is another way of “not-trusting.” I think checklists are great. It isn’t that I don’t trust people to try and do the right thing. I just don’t trust people alone, when systems can be designed with verification that improves performance. I hear people complaign that checklists “don’t respect my expertise” or have the attitude that they are “insulting to me as a professional” – you should just trust me.

Sorry, driving out fear (and building trust – one of Deming’s 14 points) is not about catering to every person’s desire. For Deming’s System of Profound Knowledge: respect for people is part of a system that requires understand variation and systems thinking and an understanding of psychology and theory of knowledge. Checklists (and other forms of verification) are not an indication of a lack of trust. They are a a form of process measure (in a way) that has been proven to improve results.

Continue reading

2011 Management Blog Roundup: Stats Made Easy

The 4th Annual Management blog roundup is coming to a close soon. This is my 3rd and final review post looking back at 2001, the previous two posts looked at: Gemba Panta Rei and the Lean Six Sigma Blog.

I have special affinity for the use of statistics to understand and improve. I imaging it is both genetic and psychological. My father was a statistician and I have found memories of applying statistical thinking to understand a result or system. I also am comfortable with numbers, and like most people enjoy working with things I have an affinity for.

photo of Mark Anderson

Mark Anderson

Mark Anderson’s Stats Made Easy blog brings statistical thinking to managers. And this is not an easy thing to do, as one of his posts shows, we have an ability to ignore data we don’t want to know. Wrong more often than right but never in doubt: “Kahneman examined the illusion of skill in a group of investment advisors who competed for annual performance bonuses. He found zero correlation on year-to-year rankings, thus the firm was simply rewarding luck. What I find most interesting is his observation that even when confronted with irrefutable evidence of misplaced confidence in one’s own ability to prognosticate, most people just carry on with the same level of self-assurance.”

That actually practice of experimentation (PDSA…) needs improvement. Too often the iteration component is entirely missing (only one experiment is done). That is likely partially a result another big problem: the experiments are not nearly short enough. Mark offered very wise advice on the Strategy of experimentation: Break it into a series of smaller stages. “The rule-of-thumb I worked from as a process development engineer is not to put more than 25% of your budget into the first experiment, thus allowing the chance to adapt as you work through the project (or abandon it altogether).” And note that, abandon it altogether option. Don’t just proceed with a plan if what you learn makes that option unwise: too often we act based on expectations rather than evidence.

In Why coaches regress to be mean, Mark explained the problem with reacting to common cause variation and “learning” that it helped to do so. “A case in point is the flight instructor who lavishes praise on a training-pilot who makes a lucky landing. Naturally the next result is not so good. Later the pilot bounces in very badly — again purely by chance (a gust of wind). The instructor roars disapproval. That seems to do the trick — the next landing is much smoother.” When you ascribe special causation to common cause variation you often confirm your own biases.

Mark’s blog doesn’t mention six sigma by name in his 2011 posts but the statistical thinking expressed throughout the year make this a must for those working in six sigma programs.

Related: 2009 Curious Cat Management Blog Carnival2010 Management Blog Review: Software, Manufacturing and Leadership

Eliminate the Waste of Waiting in Line with Queuing Theory

One thing that frustrates me is how managers fail to adopt proven strategies for decades. One very obvious example is using queuing theory to setup lines.

Yes it may be even better to adopt strategies to eliminate as much waiting in line as possible, but if there is still waiting in line occurring and you are not having one queue served by multiple representatives shame on you and your company.

Related: Customer Focus and Internet Travel SearchYouTube Uses Multivariate Experiment To Improve Sign-ups 15%Making Life Difficult for Customers

Steve Jobs Discussing Customer Focus at NeXT

Video from 1991 when Steve Jobs was at NeXT. Even with the customer focus however, NeXT failed. But this does show the difficulty in how to truly apply customer focus. You have to be creative. You have examine data. You have to really understand how your customers use your products or services (go to the gemba). You have to speculate about the future. The video is also great evidence of providing insight to all employees of the current thinking of executives.

Related: Sometimes Micro-managing Works (Jobs)Delighting CustomersWhat Job Does Your Product Do?

One factor at a time (OFAT) Versus Factorial Designs

Guest post by Bradley Jones

Almost a hundred years ago R. A. Fisher‘s boss published an article espousing OFAT (one factor at a time). Fisher responded with an article of his own laying out his justification for factorial design. I admire the courage it took to contradict his boss in print!

Fisher’s argument was mainly about efficiency – that you could learn as much about many factors as you learned about one in the same number of trials. Saving money and effort is a powerful and positive motivator.

The most common argument I read against OFAT these days has to do with inability to detect interactions and the possibility of finding suboptimal factor settings at the end of the investigation. I admit to using these arguments myself in print.

I don’t think these arguments are as effective as Fisher’s original argument.

To play the devil’s advocate for a moment consider this thought experiment. You have to climb a hill that runs on a line going from southwest to northeast but you are only allowed to make steps that are due north or south or due east or west. Though you will have to make many zig zags you will eventually make it to the top. If you noted your altitude at each step, you would have enough data to fit a response surface.

Obviously this approach is very inefficient but it is not impossible. Don’t mistake my intent here. I am definitely not an advocate of OFAT. Rather I would like to find more convincing arguments to persuade experimenters to move to multi-factor design.

Related: The Purpose of Factorial Designed ExperimentsUsing Design of Experimentsarticles by R.A. Fisherarticles on using factorial design of experimentsDoes good experimental design require changing only one factor at a time (OFAT)?Statistics for Experimenters

Warren Buffett’s 2010 Letter to Shareholders

Warren Buffett has published his always excellent annual shareholder letter. His letters, provide excellent investing insight and good management ideas.

Yearly figures, it should be noted, are neither to be ignored nor viewed as all-important. The pace of the earth’s movement around the sun is not synchronized with the time required for either investment ideas or operating decisions to bear fruit. At GEICO, for example, we enthusiastically spent $900 million last year on advertising to obtain policyholders who deliver us no immediate profits. If we could spend twice that amount productively, we would happily do so though short-term results would be further penalized. Many large investments at our railroad and utility operations are also made with an eye to payoffs well down the road.

At Berkshire, managers can focus on running their businesses: They are not subjected to meetings at headquarters nor financing worries nor Wall Street harassment. They simply get a letter from me every two years and call me when they wish. And their wishes do differ: There are managers to whom I have not talked in the last year, while there is one with whom I talk almost daily. Our trust is in people rather than process. A “hire well, manage little” code suits both them and me.

Cultures self-propagate. Winston Churchill once said, “You shape your houses and then they shape you.” That wisdom applies to businesses as well. Bureaucratic procedures beget more bureaucracy, and imperial corporate palaces induce imperious behavior. (As one wag put it, “You know you’re no longer CEO when you get in the back seat of your car and it doesn’t move.”) At Berkshire’s “World Headquarters” our annual rent is $270,212. Moreover, the home-office investment in furniture, art, Coke dispenser, lunch room, high-tech equipment – you name it – totals $301,363. As long as Charlie and I treat your money as if it were our own, Berkshire’s managers are likely to be careful with it as well.

At bottom, a sound insurance operation requires four disciplines… (4) The willingness to walk away if the appropriate premium can’t be obtained. Many insurers pass the first three tests and flunk the fourth. The urgings of Wall Street, pressures from the agency force and brokers, or simply a refusal by a testosterone-driven CEO to accept shrinking volumes has led too many insurers to write business at inadequate prices. “The other guy is doing it so we must as well” spells trouble in any business, but none more so than insurance.

I don’t agree with everything he says. And what works at one company, obviously won’t work everywhere. Copying doesn’t work. Learning from others and understanding what makes it work and then determining how to incorporate some of the ideas into your organization can be valuable. I don’t believe in “Our trust is in people rather than process.” I do believe in “hire well, manage little.” Exactly what those phrases mean is not necessarily straight forward. I believe you need to focus on creating a Deming based management system and that will require educating and coaching managers about how to manage such a system. But that the management decisions about day to day operations should be left to those who are working on the processes in question (which will often be workers, that are not managers, sometimes will be supervisors and managers and sometimes will be senior executives).

Related: Too often, executive compensation in the U.S. is ridiculously out of line with performance.Management Advice from Warren BuffetGreat Advice from Warren Buffett to University of Texas – Austin business school students2004 Warren Buffet Report
Continue reading

  • Recent Trackbacks

  • Comments