Tag Archives: Software Development

Deming and Software Development

I am sometimes asked about how use Deming’s ideas on management in a software development context. My belief is Deming’s ideas work extremely well in a software development context. The main issue is often unlearning some assumptions that people might have about what the Deming management system is.

It really is surprising to me how many “knowledge workers” respect Deming ideas but then say his attempts to treat factory workers as thoughtful people who should be respected and involved in improving their processes doesn’t make sense for them because they are “knowledge workers.”

There are many good things being done to improving the software development process. I think many of them are very Deming-like in their approaches (but to me miss out on aspects of the Deming management system that would be helpful). I think Dr. Deming’s approach to software development would focuses on the system of profound knowledge (the 4 inter-related areas below):

Understanding variation – software development has quite a bit of variation, some probably innate [unique work] and some due to not having good procedures, batching work, not fixing problems right when they are seen, quick fixes that leave the system vulnerable in the long term (when you make one simple change to the code it has an unanticipated consequence due to poor practices that could have been eliminated), etc.. Many good coding practices are effective strategies to deal with this issue. And building an understanding of variation for managers (and business process owners/product owners) is very helpful to the software development process. The ideas in agile and kanban of focusing on smaller delivery units of work (one piece flow, just in time, cycle time…), customer value, maintainable code, sustainable work conditions, etc. are directly found in a Deming management system.
Appreciation for the system of software development. Don’t just complain about bugs. Examine the process of development and then put in place mistake proofing efforts (don’t duplicate code, use integrated regression tests, don’t put artificial constraints on that result in system distortions – unrealistic targets…). Use things like kanban, limited work in progress, delivering value to customers quickly, think of success in terms of getting working software to customers (not meeting internal delivery goals), etc. that take into account our experience with systemic software development problems over the decades.
Theory of knowledge – how do we know what we know? Are estimates reliable? Lets look at what users do, not just what they say (A/B testing…). Software developers often appreciate the value of usability testing, even though they rarely work for organizations willing to invest in usability testing. In my experience when software developers object to usability testing it is normally really an objection to overwork, and the usability testing is just going to give them more work or criticize things they were not allowed to spend the time they needed to do great work. That won’t always be the reason but it is the main one in my experience (I suppose their is also fear and just the psychology of not wanting to hear anything negative about what has been created – even if the usability testing shows tons of great results people will often focus on the negative).
psychology and respect for people – This pretty much seems like it is the same for software development as everywhere else.

Continue reading →

What is the Explanation Going to be if This Attempt Fails?

Occasionally during my career I have been surprised by new insights. One of the things I found remarkable was how quickly I thought up a new explanation for what could have caused a problem when the previously expressed explanation was proven wrong. After awhile I stopped finding it remarkable and found it remarkable how long it took me to figure out that this happened.

I discovered this as I programmed software applications. You constantly have code fail to run as you expect and so get plenty of instances to learn the behavior I described above. While I probably added to my opportunities to learn by being a less than stellar coder I also learned that even stellar coders constantly have to iterate through the process of creating code and seeing if it works, figuring out why it didn’t and trying again.

The remarkable thing is how easily I could come up with an new explanation. Often nearly immediately upon what I expected to work failing to do so. And one of the wonderful things about software code is often you can then make the change in 10 minutes and a few minutes later see if it worked (I am guessing my brain kept puzzling over the ideas involved and was ready with a new idea when I was surprised by failure).

When I struggled a bit to find an initial explanation I found myself thinking, “this has to be it” often because of two self reinforcing factors.

First, I couldn’t think of anything else that would explain it. Sometimes you will think right away of 4 possible issues that could cause this problem. But, when I struggled to find any and then finally came up with an idea it feels like if there was another possibility I should have thought of it while struggling to figure out what I finally settled on.

Second, the idea often seems to explain exactly what happened, and it often feels like “of course it didn’t work, what was I thinking I need to do x.” This often turns out to be true, doing x solves the problem and you move on. But a remarkable percentage of the time, say even just 10%, it doesn’t. And then I would find myself almost immediately thinking, of course I need to do y. Even when 10 seconds ago I was convinced there was no other possibility.

Continue reading →

Software Process and Measurement Podcast With John Hunter

In my podcast with Tom Cagley, Software Process and Measurement Cast: John Hunter on Management Matters, as you might expect there was a bit of a focus on software development and agile software development as related to the ideas I expressed in Management Matters: Building Enterprise Capability.

Deming: management system, continual improvement, respect for people (the 3rd point I said I would mention, but didn’t)
Why do so many management improvement efforts fail?
The proxy nature of data
Building Adoption of Management Improvement Ideas in Your Organization

photo of John Hunter at the Borobudur Temple

John Hunter at the Borobudur Buddhist Temple in Indonesia.

Continue reading →

Podcast Discussion on Management Matters

I continue to record podcasts as I promote my new book – Management Matters: Building Enterprise Capability. This the second part, of 2, of my podcast with Joe Dager, Business 901: Management Matters to a Curious Cat. The first part featured a discussion of 2 new deadly diseases facing companies.

image of the cover of Managmenet Matters by John Hunter

Management Matters by John Hunter

Listen to this podcast.

Links to more information on some of the topics I mention in the podcast:

Appreciation for a System
Using the PDSA improvement cycle
Dangers of Targets Distorting the System
Clayton Christensen and others
Influencing your organization: Building Adoption of Management Improvement Ideas in Your Organization – Grow Your Circle of Influence – How to Get a New Management Strategies, Tools and Concepts Adopted
Psychology: Respect for Everyone – Managing people – confirmation bias – Fooled by Randomness

More podcasts: Process Excellence Network Podcast with John Hunter – Business 901 Podcast with John Hunter: Deming’s Management Ideas Today (2012) – Leanpub Podcast on Management Matters: Building Enterprise Capability

Joy in Work in the Quality Improvement Field

As I mentioned previously, I will be posting on a topics raised by Paul Borawski, CEO, ASQ as part of ASQ Influential Voices. This month Paul’s post, Are Quality Professionals Happy On the Job? looks at job happiness in the quality improvement field.

Paul stated he “wasn’t surprised that Forbes Magazine named software quality assurance engineer as the ‘happiest job’ in the U.S.” I was. Frankly looking at the results I question the methodology used – I just find their claims questionable. Whether any ranking could be sensible is also a reasonable question. I do believe certain careers make people happier than others, I question whether you can sensibly differentiate the top 20.

Still, looking at the happiness of those in the quality field is an interesting topic. My father created a challenge for me. He loved what he did: professor (statistics, chemical engineer, industrial engineer, business) and consultant (same things, with focus on quality and management improvement). Helping achieve better results was important to him. And helping create joy in work was also. It took me a while to see how much of an outlier he was – finding people who love what they do is much rarer than those that complain a great deal I have found.

That software development ranks toward the top doesn’t surprise me. Software programmers are some of the people happiest in their jobs in my experience. My experience is biased toward those given more freedom than those working in large bureaucracies (I can imagine those programmers are less happy overall). In addition to being happier with their jobs they also are demanding. They are not the least challenging of authority (some managers seem to equate docility with happiness, but that isn’t accurate, in my opinion).

To me the quality field allows for a great deal of joy in work. That doesn’t mean it is without frustration. I think the field does have a fairly high level of frustration as many are stuck in systems that are moving much to slowly to improve management practices. This is the biggest concern I find from most in the quality improvement field. So in order to be happy one has to learn to cope with some frustration while making good progress and finding happiness in all the achievements even while knowing more could be done.

Continue reading →

Respect for People: Optimize for Developer Happiness at Etsy

The webcast above discusses the culture of software engineering at Etsy (a very popular site providing a marketplace and community for small businesses – artisan focus). Some of the key points of the talk. Etsy trusts employees. Etsy’s strategy is to optimize for developer happiness. Etsy has lunches twice a week where employees build community.

Etsy sees code as craft. The echos Etsy’s value on authorship: “the people behind what we buy make commerce meaningful.” It re-inforces the belief that work has meaning and is valued and should have intrinsic value to those doing the work, people should have the opportunity to take pride in their work.

Chad Dickerson discussed the importance Peter Drucker placed on connecting people to the value provided to customer. Etsy takes steps to connect employees to the value provided to customers, including emphasizing the community of the company and the customers of Etsy.

Continue reading →

Management Improvement Blog Carnival #160

Monkey at the Singapore Zoo by John Hunter

The Curious Cat Management blog carnival highlights recent management blog posts 3 times each month. The posts generally focus on the areas I have focused on in the Curious Cat Management Guide since 1996 (Deming, innovation, lean manufacturing, customer focus, process improvement…).

Reflections on the 100th Birthday of Taiichi Ohno by Masaaki Imai – “Taiichi Ohno always placed respect for the worker first in his approach to kaizen. His focus was always on the customer, both external and internal”
A Lean Leader strengthens the business by developing people through coaching process improvement at the gemba by Jeff Liker – “Thinking of a leader as a teacher and coach, as managing from the gemba, believing deeply that people are the only appreciating assets of the company, believing in the value of intentionally creating a common culture and being a role model of that culture, and that the adaptiveness of the business to meet the challenges of the environment comes from how people are developed all the way down to the worker is quite different than the leader as the captain of the ship steering it cleverly through brilliant personal insights.”
Inspiration Stimulates Productivity and Engagement by Nicole Radziwill – “I’d also like to propose that engagement is a symptom – a consequence of feeling good and having a high quality consciousness! Let’s work on the root causes, and focus less on the symptoms.”
Kanban Networks Exerciseby Yuval Yeret – “The exercise brought to life the complexity of the organization’s network but highlighted how a Kanban system can simplify its operation as well as drive towards improvement. There were several A-Ha moments of understanding how Limited WIP would solve systemic problems currently haunting the organization.”

Continue reading →

Learn to Code to Help Your Career

I believe there are big benefits to knowing how to code (programing, software development). What is possible for your organization is often significantly impacted by understanding how to properly use software (and create it, coding, when needed). The lack of understanding of software is a significant problem not just for those wanting a job coding (that are available for those with the right skills) but also for those making decisions about what the organization should do.

The profound ignorance (meant not in a pejorative way but in the descriptive way) of software is a significant problem for managers today. The critical role of software in our organizations is only growing. And the importance of understanding software (which coding provides in a way no other learning does) is only increasing. My guess is a decade or two or three from now a understanding of coding will not be nearly as critical for managers. I am just guessing the nature of coding will be significantly changed and not understanding the details needed to code will not be as critical as it is today. Maybe I am wrong about the importance of understanding coding fading over time (it is more a feeling than a chain of logic I can clearly explain easily).

There are many indirect benefits of learning to code. In the same way that those with an education in engineering do very well in their careers overall, even if they take a path where they are no longer engineers a background in coding prepares you well for your career. Actually, similar to engineering, part of this effect may well be those that can graduate with an engineering degree and those that can be employed for several years as a software developer have skills and abilities that would have made them successful even if they didn’t pass through those experiences (still I think, those experiences to add to their success).

Good programmers have a strong tendency to think in ways that those interested in management improvement need (and, sadly, often lack): systems thinking, customer focus, efficiency focused [good coders often hate wasting their time and naturally despise non-value added steps], a willingness to speak up about things that need to be improved, a desire to make a difference, passion for what they do…

If you work along side good programmers these traits will be reinforced every day (this was my favorite part of my last job – working with great programmers that pursued these principles and re-enforced my doing so also). Yes there are also things you might have to temper in dealings with non-coders (being a bit kinder/less-direct about perceived failures, for example). Also some coders can be so engaged they expect an unsustainable commitment from peers (this is one of the great benefits of a good agile software development system – a focus on creating an environment for sustainable development [not expecting unreasonable effort/hours on the part of coders]).

Continue reading →

Agile Story Point Estimation

In agile software development tasks are documented as user stories. Then the level of effort for those stores can be estimated by assigning each story points. The velocity that can be produced in a period (called a sprint, for us 2 weeks) can be estimated. Thus you can predict what can be delivered in the next sprint (which can help business managers make priority decisions).

I have found estimation to be worthwhile. In doing so, we accept there is a great amount of variation but points give a hint to scale. They can help prioritize (if you have 5 things you want but 1 is much harder you may well drop that to the bottom). I have always accepted a great amount of variation in the velocity, worry about the variation I don’t find worthwhile. I do think trying to act as though the velocity is precise can lead to problems. At the same time having a measure of velocity, even accepting understanding variation was present, was useful.

Over time reducing variation (probably largely through better estimation and perhaps a few better tools, reduced technical debt, better documentation, testing…) is helpful and laudable. We made improvement but still lots of variation existed. The biggest help in reducing the measured velocity was breaking down large stories to more manageable sizes. The challenge of estimating user stories, I suspect, has some fairly high variation (even with good system improvements that can help reduce variation).

Large stories just can hide huge variation in what is really required once getting into implementing it.

The way we did estimation (discussing in a sprint planning meeting) did take some time (but not a huge amount). It was agreed by those involved that the time spent was worthwhile. Sometimes we did slip and spend too much time on this, that was an area we had to pay attention to. The discussions were educational and helped provide guidance on how to approach the story. The value of discussions around estimations was probably the biggest surprise I have had in implementing any agile ideas. The value of those discussion was much higher than I imagined (I basically anticipated them just as non-value added time to get the result of an estimate, but they were a source of learning and consensus building).

These thoughts were prompted by: Story Points Considered Harmful – Or why the future of estimation is really in our past…

Continue reading →

Management Improvement Blog Carnival #156

The Curious Cat Management blog carnival highlights recent management blog posts 3 times each month. The posts generally focus on the areas I have focused on in the Curious Cat Management Guide since 1996 (Deming, evidence based management, lean manufacturing, agile software development, systems thinking…)

The Key Questions for a Minimum Viable Product Project by Anthony Panozzo – “What are you trying to learn with this particular MVP?
What data are you collecting about your experiment?
What determines the success or failure of the experiment?” [bold added – John]
Less Process, More Discipline by Charlie Martin – “Without it, you lose everything agile methods promise. The key to agile methods is this: You may have less process, but you must have more discipline.”

Sunset over Andaman, Khao Lak, Thailand. By John Hunter

Evaluating Executive Performance by Art Smalley – “One interesting thing that I will note that was considered in Toyota in Japan by the HR department when evaluating executives was how their previous departments fared after they had left. If the department continued to improve then this was generally a good sign.”
The evolution of design to amplify flow by John Hagel – “If we want to remain successful and reap the enormous rewards that can be generated from flows, we must continually seek to refine the designs of the systems that we spend time in to ensure that they are ever more effective in sustaining and amplifying flows.”

Continue reading →