Mistake Proofing Deployment of Software Code

This is a continuation of my previous post: Improving Software Development with Automated Tests. Lets look at a typical poka-yoke example. A USB connector must be put in the right way up – for the connection to work properly and the communication to occur as intended. So to mistake proof the process the connector won’t allow the USB device to be put in upside down – the hardware connection designed to not allow that type of connection.

Using a deployment process that prevents code from being submitted that has an error follows a nearly identical process. The process blocks an error from being made. It seems to me a process that blocks code with a bug from being deployed with an error is the basically the same as a USB connection that will not accept the device being put in upside down.

Mistake proofing in no way should limit focusing on improving the process. Mistake-proofing a process both improves it (many poka-yoke solutions make the process easier to use) and prevents an error in case you still try something wrong. So I see the automated tests as a way to serve as a backstop, in case the process improvement you made to the software development process failed in some form. Then the automated testing required to deploy would prevent the introduction of that error to the production environment.

Related: Checklists in Software DevelopmentBaking in Quality to Software DevelopmentCombinatorial Testing for Software DevelopmentGreat Visual Instruction Example

Posted in Management, Quality tools, Software Development | Tagged , , , , | 6 Comments

Combinatorial Testing – The Quadrant of Massive Efficiency Gains

My brother, Justin Hunter, gives a lightning talk on Combinatorial Testing – The Quadrant of Doom and The Quadrant of Massive Efficiency Gains in the video above. The following text is largely directly quoted from the talk – with a bit of editing by me.

When you have a situation that has many many many possible parameters and each time only a few possible choices (a few items you are trying to vary and test – in his example in the video, 2 choices) you wind up with a ridicules number of possible tests. But you can cover all the possibilities in just 30 tests if your coverage target is all possible pairs. When you have situations like that you will see dramatic efficiency gains. What we have found in real world tests is greatly reduced time to create the tests and consistently 2 to 3 times as many defects found compared to the standard methods used for software testing.

You can read more on these ideas on his blog, where he explores software testing and combinatorial testing. The web base software testing application my brother created and shows in the demo is Hexawise. It is free to try out. I recommend it, though I am biased.

Related: Combinatorial Testing for SoftwareVideo Highlight Reel of Hexawise – a pairwise testing tool and combinatorial testing toolYouTube Uses Multivariate Experiment To Improve Sign-ups 15%What Else Can Software Development and Testing Learn from Manufacturing? Don’t Forget Design of Experiments (DoE)Maximize Test Coverage Efficiency And Minimize the Number of Tests Needed

Justin posted the presentation slides online at for anyone who is interested in seeing more details about the test plan he reviewed that had 1,746,756,896,558,880,852,541,440 possible tests. The slides are well worth reading.
Continue reading

Posted in Quality tools, Software Development | Tagged , , , , | 3 Comments

Management Blog Posts From May 2006

gold figure from South America at the Met

I think looking back at some previous posts to the blog is worthwhile. Here are links to some of the posts from May 2006. The photo shows some South American artwork at the Metropolitan Museum of Art: I took the photo on a trip to New York.

  • Using Data Effectively Requires Thought – The numbers are just proxies for the real situation. Look beyond the numbers you see to what they mean and understand how the numbers presented may not fully capture the important details you need to consider.
  • Better and Different – Toyota is a great example of doing both. So are Google and Apple. But doing things differently also means taking risks and Apple has suffered in the past. Doing things differently is great as long it is the right differently (which isn’t always easy to judge).
  • Find the Root Cause Instead of the Person to Blame – If you find yourself thinking a problem or defect is the fault of a person try asking why a couple more times and see if you can find a system improvement that would eliminate or mitigate such problems in the future. That is a much more effective improvement strategy.
  • Respect for People – I think we often have so little respect for people we just avoid dealing with anything touchy because we don’t want to risk they won’t be able to react to the issues raised and will instead just react as if they have been personally attacked.
  • Problems Caused by Performance Appraisal – People are increasingly challenging the notion that we just have to live with performance appraisal systems. As usually, I will make my suggestion that chapter 9 of the Leader’s Handbook offers great material on performing without appraisal
  • Lean Thinking Misconception – It is a shame so many think lean is equal to “reducing staff.” Lean is about removing waste. Removing waste might mean that fewer people are needed but no real lean thinking organization aims to reduce staff.
Posted in Management | Tagged | Comments Off on Management Blog Posts From May 2006

Interruptions Can Severely Damage Performance

Interruptions can severely degrade your performance. The type of work you are doing impacts the cost greatly. I have spent some of my time programming web applications. When I am doing that interruptions are a huge drain on my performance (for me the costs of interruptions while programming are far higher than any other type of work I have done – many times higher). If the interruption disrupts my flow (an interruption needn’t necessarily disrupt it I found, instant messages may not, while speaking to someone else almost surely would – it is a factor of how much of your brain much shift focus I imagine) it can take a huge amount of time to get back into a high performing state. Other work I do can be interrupted with much less impact. I am easily able to slip back into what I was doing.

For me the main cost of interruptions is the time it takes to get back to where I was before the interruption. And the cost is related to how much focus is needed to address what you are working on. Most programming takes a huge amount of focus.

Another big cost of interruptions is the increased risk of mistakes. When people are distracted and then have to go back to a task, and then are distracted, and then go back and… it is more likely they will miss a step or miss noticing some issue than if they can work without distraction. One tool to help cope for distractions that can’t be designed out are checklists.

Paul Graham addressed the importance of managing the system to provide uninterrupted time very well in, Maker’s Schedule, Manager’s Schedule

One reason programmers dislike meetings so much is that they’re on a different type of schedule from other people. Meetings cost them more

Most powerful people are on the manager’s schedule. It’s the schedule of command. But there’s another way of using time that’s common among people who make things, like programmers and writers. They generally prefer to use time in units of half a day at least. You can’t write or program well in units of an hour. That’s barely enough time to get started.

Paul Graham’s article also shows why managers so often fail to adequately address this issue. Managers, by and large, work in an environment where interruptions are the work. I know, much of my time as a program manager is driven by interruptions and the job is doable even with many interruptions every day. I did still find there were parts of the work that required uninterrupted time to think deeply.

When managing you need to understand how big a cost interruptions have and design systems appropriate to optimize system performance for all parts of the system. The design of the system needs to take into account the costs and benefits of interruptions for those people working on various processes in the system.

Related: Understanding How to Manage GeeksExplaining Managers to ProgrammersWhat Motivates Programmers?Joy in Work – Software DevelopmentProgrammers CartoonChecklists in Software Development

Posted in Creativity, Management, Psychology, quote, Respect, Software Development, Systems thinking | Tagged , , , , , , , , , , , | 7 Comments

Management Improvement Carnival #97

Mike Wroblewski is hosting Management Improvement Carnival #97 on the Got Boondoggle? blog, highlights include:

  • Bagel with a side of Jidoka by Evan Durant – “And the more we explore the concept of Jidoka the more we are forced to challenge our assumptions about what exactly the human and machine elements are. Often there are untapped opportunities to separate the two.”
  • The Human in the Loop by Mark Rosenthal – “If we truly want to construct a work environment where people make the best possible decisions, it behooves us to rid ourselves of decades old stereotypes and convenient beliefs about why people decide what they do.”
  • 3 Tips for Continuous Improvement by Ron Pereira – “In this article, I want to share some ideas for how to approach things such as workout programs and continuous improvement as they are surprisingly similar.”
  • Should I Pursue Waste Elimination or Lead Time Reduction? by Michael Balle as the Gemba Coach – “To respond to your question directly; there is no debate: Kaizen without a pull system will be disappointing.”

Related: Management Improvement Carnival #64Curious Cat Management Improvement Search

Posted in Carnival, Management | Tagged | 1 Comment

Manufacturing in the USA, and Why Organizations Often Don’t

Manufacturing in the USA continues to do well. But it could do better. There are reasons manufacturing that could be located in the USA is not. And addressing those can increase USA manufacturing. Some reasons are sensible, based on the existing economics and realities of comparative advantage. Some reasons are just flawed thinking, such as the “spreadsheet management” taught at many business schools that Deming and lean thinkers can understand the flawed thinking that leads to outsourcing.

Typical wall street thinking (also driven by “spreadsheet management think” rather than an actual understanding of value stream of a potential investment) also hampers raising investment capital for USA manufacturing. The broken USA health care system also is a big problem driving up costs of doing business in the USA enormously.

Fighting for ‘made in the USA’ [the broken link was removed]

Safer and longer-lasting than conventional lithium-ion car batteries, the 52-year old MIT professor’s invention packs 600 cells into a case the size of an airplane carry-on bag. His technology has transformed the batteries used in many cordless power tools. So why are Chiang and his company, A123 Systems, having trouble moving to full-scale commercial production and creating thousands of new American jobs with his better mousetrap?

Despite the promise of Chiang’s batteries, many on Wall Street and in Silicon Valley were incredulous when he and other leaders at A123 asked for capital to build factories in America — Asia, yes, but Michigan, why would you want to?

Even more daunting, nearly all of the world’s battery manufacturing industry is in Asia, where plants can be built faster and supplies and equipment are much easier to get than in the United States. These days, it’s hard to find Americans who even know how to build a battery factory.

That’s why A123 had to give in and build its first plants in China, where the company could move into production quickly to show auto industry customers that it could deliver on future contracts.

“Without question, we would rather have done it all in the U.S.,” said Chiang, who left Taiwan as a 6-year-old with his family, earned degrees at MIT and has been a materials science professor there since the mid-1980s. “I’m an American citizen. We’re an American company. It’s an American-born technology.”

Despite the obstacles, A123 and a few other advanced battery producers are building plants in Michigan and other states, thanks to massive government support that has offset Wall Street’s skepticism and should help domestic producers narrow cost disadvantages with Asian rivals.

A123 is getting $250 million in aid from the Obama administration’s stimulus program as well as tax incentives from Michigan. Its first U.S. plant opens in June in an abandoned brick building near Detroit that once made VHS tapes for Disney.

A123 has five plants in China, coincidentally located in Chiang’s father’s hometown of Changzhou, about two hours’ drive west of Shanghai. Bart Riley, an A123 co-founder and chief technology officer, figured it took about nine months to get a Chinese factory up and running, one-third the time typical for the U.S.

The quicker launch helped A123 make a name for itself through Black & Decker, which in early 2006 began putting A123 batteries in its DeWalt power tools.

Since then, A123 has been supplying batteries and battery systems for New York City buses built by Daimler, among other customers, and the company has agreements to develop products for Chrysler, Navistar and American green-car maker Fisker Automotive.

By the end of next year, A123 expects to have two plants in Michigan employing 400 people, with plans to go up to 2,000 workers able to produce about 30,000 battery systems a year. The company’s sales reached $91 million last year, and it has about 1,700 employees, two-thirds in Asia.

The success of science and engineering university based research is still a huge advantage to the USA. Though other countries have seen the value in this and have invested in building their own capacity. The economic value of such is increased many fold by manufacturing the innovations created in your country.

Related: Manufacturing and the EconomyEconomic Strength Through Technology LeadershipRhode Island ManufacturingBig Failed Three, Meet the Successful EightToyota in the US Economy

Posted in Economics, Manufacturing | Tagged , , , | 3 Comments

Ignoring Unpleasant Truths is Often Encouraged

We can’t Handle the Truth

Unfortunately, the proverbial “kill the messenger” is alive and well in American business. People who speak the truth are often labeled as a non-team player, a disrupter, a trouble maker or the current tag of being “not a good fit”.

It doesn’t take much to see that the truth can get watered down, altered or hidden entirely inside a company, especially as it moves vertically up the ladder. We may believe, at least in the short term, that this is the best way considering the risk, political correctness and social politeness but at what cost?

What I have seen is truth is not valued much. I’m interested in creating improvement. I thought people would be driven by data and possible strategies to improve. But I have found that just isn’t often true.

So, from my experience, the strategy to improve means not distracting people with many of the truths. Try to fix the system and convince others to fix the system as you can. If some efforts are resisted try to adjust. Sometimes try a different strategy to get improvement. Sometimes I just drop trying to improve that particular thing. There are usually so many options for improvement it isn’t tough to find plenty of others that may be more successfully tackled.

I am someone who find it frustrating that many don’t seem interested in really understanding what the system is producing and where weaknesses exist. But, at least for me, trying to have things work the way I want (where an open exploration of the truth was the focus) isn’t the priority. I have figured it is better to give up on that desire and work within the reality that exists.

I can’t stop myself though from pointing out things far more often than people want. I have no doubt it has annoyed people and gotten me in trouble. But nothing that wasn’t manageable, I just make things a bit more difficult for myself.

One, very visible, sign of people avoiding the truth is if people say very different things in meetings and outside of them. It is amazing to me how much less likely it is for anything that could be seen as a complaint or criticism to be voiced in a meeting versus in the hallway. It isn’t so surprising if you understand human psychology (the tendency to blame those who voice a problem).

People figure this out and keep their mouth shut. But to their friends they understand they can point out the problems and not be blamed (so in the hallways, you get a much more honest view of what people think). This is a bad sign. If your organization trains people to ignore unpleasant truths it makes managing more difficult and results in poorer performance.

Related: The difference between respect and disrespect is not avoiding avoiding criticismInformation Technology and Business Process SupportHow to ImproveFind the Root Cause Instead of the Person to BlameInformation Technology and ManagementRespect for People, Understanding PsychologyBring Me Problems, and Solutions if You Have ThemBetter MeetingsPeople are Our Most Important Asset

Posted in Management, Process improvement, Psychology, quote, Systems thinking | Tagged , , , , , | 6 Comments

Curious Cat Management Carnival #96

The Curious Cat Management blog carnival selects recent management blog posts 3 times each month. Also visit the Curious Cat Management Library for online management improvement articles.

  • Kanban Systems in Software Development – “In the field, I’ve seen Kanban work best in chaotic environments where upcoming features don’t have much in common.”
  • Renegade Recruiting by Chris Ferdinandi – “I think world-class recruiting is really about three things: Building a pipeline of qualified talent before you need to hire someone. Accurate, valid selection criteria. A fantastic candidate experience.”
  • How to Deal with Complainers – 2 Approaches by Harwell Thrasher – “People who complain fall into two categories: those who complain because they want help in resolving a problem, and those who complain because they want sympathy. Often the complainers themselves don’t understand why they’re complaining, so it’s up to you to figure it out for yourself.”
  • Building Coaching Capabilities
    by Lee Fried – “Let’s be clear the purpose of coaching is to build capability into the leadership line.”
  • How to Deal with Low Performers by Tim McMahon – “A production lead should use this simple 5 step checklist… 3) Has the person been trained?… 5) Has there been regular feedback on performance? An answer of “no” to any of these questions will indicate an area for which focused improvement is needed.”
  • Top Three Motivators For Developers (Hint: not money!) by Dave Rodenbaugh – “Every developer on the planet wants to get better at what they do. We crave new knowledge like some people quaff coffee after a hangover… Nothing is more tedious, horrific, or uninspiring to developers to work on projects that lack any real meaning in the world.” (this is another in a long line of posts about Danial Pink’s Ted Talk – John).
  • We can’t Handle the Truth by Mike Wroblewski – “People who speak the truth are often labeled as a non-team player, a disrupter, a trouble maker or the current tag of being ‘not a good fit’. End result the person either quits or is fired.”
  • Get A Life (Not A Job) by Mark Stelzner – “Great career acts: They all share five key elements – 1) Self-awareness; 2) Continuous self-development; 3) Unique and critical roles; 4) Well-managed time, money and human resources; and 5) Harmony among your work, family and personal life” – buy the book
  • Continue reading

Posted in Carnival | Tagged , | 1 Comment

Nice Non-technical Control Chart Webcast

This very brief introduction to control charts by PQ Systems provides a very watchable non-technical overview. Getting people to understand variation is important, and not easy. This video is one more quick reminder for those still trying to incorporate an understanding of variation into their view of the world.

The idea is simple. But actually thinking with an understanding of variation people find difficult, it seems to me. It is very easy to continue to revert to special cause thinking (who did it? is often a sign of special cause thinking) – thinking that results are due to a special (unique) cause, instead of as the result of a system (which includes lots of common causes).

The value I see in this video is as a reminder for all those trying to operate with an understanding of variation. It is also a decent introduction, but much, much more would be needed to get people to understand why this matters and what is needed.

Related: Control Charts in Health CareHow to Create a Control Chart for Seasonal or Trending DataMeasurement and Data CollectionSix Sigma and Common SenseEuropean Blackout, not Human Error

Posted in Deming, Management, Process improvement, Quality tools, Systems thinking, webcast | Tagged , , , , , , | 1 Comment

Management Improvement Carnival #95

Bryan Lund is hosting Management Improvement Carnival #95 on the Training Within Industry blog, highlights include:

  • Kathleen Fasenella talks about “How to Sew Faster” at Fashion Incubator… Leansters will appreciate the industrial engineer that resides in Kathleen’s work – Part 4 of her “How to Sew Faster” series talks about the 4Ms: Man, Machine, Material and Machines. Kathleen’s posts often remind me of Taichii Ohno’s writings
  • Bryan Zeigler at Lean is Good dredges up an old Deming experiment to illustrate a common management malpractice: Chasing Rabbits and Process Unimprovements. I like this post because the experiment helps people realize the chaos that can result in their reactive behavior as managers.
  • Mark Rosenthal at The Lean Thinker makes a distinction between The Expert vs. The Master in his post Knowing vs. Knowing How to Learn. The big deal here is that Mark argues some experts will try to fit their circumstances to a specific tool, but masters distinguish themselves by learning how to go beyond the tool and apply the concept in circumstances with which they are unfamiliar.

Related: 2010 year in reviewManagement Improvement Carnival #61

Posted in Carnival, Management | Tagged | 1 Comment