Category Archives: Software Development

Respect for People: Optimize for Developer Happiness at Etsy

The webcast above discusses the culture of software engineering at Etsy (a very popular site providing a marketplace and community for small businesses – artisan focus). Some of the key points of the talk. Etsy trusts employees. Etsy’s strategy is to optimize for developer happiness. Etsy has lunches twice a week where employees build community.

Etsy sees code as craft. The echos Etsy’s value on authorship: “the people behind what we buy make commerce meaningful.” It re-inforces the belief that work has meaning and is valued and should have intrinsic value to those doing the work, people should have the opportunity to take pride in their work.

Chad Dickerson discussed the importance Peter Drucker placed on connecting people to the value provided to customer. Etsy takes steps to connect employees to the value provided to customers, including emphasizing the community of the company and the customers of Etsy.

Related: Respect People by Creating a Climate for Joy in WorkMistake Proofing Deployment of Software CodeBuild an Environment Where Intrinsic Motivation Flourishes

Continue reading

Agile Story Point Estimation

In agile software development tasks are documented as user stories. Then the level of effort for those stores can be estimated by assigning each story points. The velocity that can be produced in a period (called a sprint, for us 2 weeks) can be estimated. Thus you can predict what can be delivered in the next sprint (which can help business managers make priority decisions).

I have found estimation to be worthwhile. In doing so, we accept there is a great amount of variation but points give a hint to scale. They can help prioritize (if you have 5 things you want but 1 is much harder you may well drop that to the bottom). I have always accepted a great amount of variation in the velocity, worry about the variation I don’t find worthwhile. I do think trying to act as though the velocity is precise can lead to problems. At the same time having a measure of velocity, even accepting understanding variation was present, was useful.

Over time reducing variation (probably largely through better estimation and perhaps a few better tools, reduced technical debt, better documentation, testing…) is helpful and laudable. We made improvement but still lots of variation existed. The biggest help in reducing the measured velocity was breaking down large stories to more manageable sizes. The challenge of estimating user stories, I suspect, has some fairly high variation (even with good system improvements that can help reduce variation).

Large stories just can hide huge variation in what is really required once getting into implementing it.

The way we did estimation (discussing in a sprint planning meeting) did take some time (but not a huge amount). It was agreed by those involved that the time spent was worthwhile. Sometimes we did slip and spend too much time on this, that was an area we had to pay attention to. The discussions were educational and helped provide guidance on how to approach the story. The value of discussions around estimations was probably the biggest surprise I have had in implementing any agile ideas. The value of those discussion was much higher than I imagined (I basically anticipated them just as non-value added time to get the result of an estimate, but they were a source of learning and consensus building).

Related: Assigning Story Points to Bug FixesMistake Proofing the Deployment of Software CodeChecklists in Software Development

These thoughts were prompted by: Story Points Considered Harmful – Or why the future of estimation is really in our past…

Continue reading

Rethinking or Moving Beyond Deming Often Just Means Applying More of What Dr. Deming Actually Said

Don Reinertsen – Is It Time to Rethink Deming? from AGILEMinds on Vimeo.

I feel very strongly about the value of Deming’s ideas. I am glad people challenge those ideas and try to push forward management thinking. Helping us manage organizations better (to get better results and allow people to better enjoy their jobs and lives) is why I value Deming’s ideas. To the extent we find better ideas I am very happy. I understand I will disagree with others on the best ways to manage, and believe healthy debate can be productive.

What Don Reinertsen discusses in the video, about special and common cause is not the best way to look at those ideas, in my opinion (though I would imagine it is the most common view). For data points that are common cause (within the control limits and not a special cause pattern) it is most effective to use common cause tools/thinking to improve. For indications of special cause (points outside the control limits or patterns in the data, such as continually increasing results that indicate a special cause) it is most effective to use special cause tools to improve.

This does not mean that a point outside the control limits is caused by a special cause (also know as assignable cause). It is just best to use special cause tools and thinking to address those data points (and the reason this is true is because it is most likely there is an assignable cause). The control limits do not define the nature of the point, they define the type of improvement strategy that should be used.

Don also says repeatedly that you don’t “respond to random variation” in Deming’s view. That is accurate. But then he implies this means you don’t address system performance, which is not. You work on improving systems (that are in control) by improving the system, not by responding to individual common cause data points (random variation) as if it were assignable cause variation.

The purpose of the control chart (that Shewhart developed) was to help you most effectively take action (knowing if special cause thinking, or system improvement, was the best improvement strategy). The control chart shows if the results are in control and tells you that the system is preforming consistently (and identifies a special cause so special cause tools can be used immediately, this is important because special cause improvement strategies are time sensitive). It tells you nothing about if the results are acceptable.

Continual improvement was also central to Deming’s management philosophy (based on the business value of the many improvement options available in every organization). For Deming this meant working on improving the system, if the results are in control, instead of trying to deal with finding a specific assignable cause for one data point and acting on that. If the issue is one of the system performance (no indication it is a special cause) the most effective strategy to get better results is to improve the system, rather than approach it as a special cause issue (examining individual data points, to find special items in that event to be improved). You can use special cause thinking, even where system improvement thinking would be better. It will work. It is just not very effective (improvement will be much slower) compared to focusing on system improvement.

I agree with Don that the United States mentality, not only in nuclear plants but everywhere, is to apply special cause thinking as the strategy for process improvement. This is one the areas Deming was trying to change. Deming, and I, think that setting your improvement strategy based on a common cause (system improvement) or assignable/special cause (learn what is special about that one instance) is the most effective way to achieve the best results. We believe in continual improvement. We believe that the effective way to improve, when a system is in statistical control, is by focusing on the whole systems (all the data) not assignable cause (special cause) thinking where you look at what is special about that bad (or good) individual result.

The economic consideration of whether the costs of improvements are worth the benefit is sensible (and I do not see Dr. Deming arguing against that). That is separate from the best method to improve. For Deming the best method to improve means using special cause thinking for assignable cause issues and common cause thinking for systems issues.

The idea of where to focus improvement efforts is not something Dr. Deming made as clear as he could have, in my opinion. So I see the argument of Deming not prioritizing where improvement should occur voiced occasionally. This is a weakness in Deming’s content, I believe, more than his philosophy (but I can understand it causing some confusion).
Continue reading

Steve Jobs Discussing Customer Focus at NeXT

Video from 1991 when Steve Jobs was at NeXT. Even with the customer focus however, NeXT failed. But this does show the difficulty in how to truly apply customer focus. You have to be creative. You have examine data. You have to really understand how your customers use your products or services (go to the gemba). You have to speculate about the future. The video is also great evidence of providing insight to all employees of the current thinking of executives.

Related: Sometimes Micro-managing Works (Jobs)Delighting CustomersWhat Job Does Your Product Do?

Avoiding Tragedy of the Commons for Software Development

Kanban and Tragedy of the Commons

The “Tragedy of the Commons” archetype often manifests itself through “Shared Services”, when a small number of people with specific skills, work across different teams. Each team in isolation gets benefit from the Shared Service, but when demand for the service exceeds its capacity, then nobody benefits. At a smaller scale, a team with a low “bus factor”, or a hero, can also suffer from a tragedy of the commons, when too much work is dependent on a single person.

One of the comments on this post suggested that the tragedy of the commons wasn’t an accurate description. My comment:

I think the “tragedy of the commons” analogy works. As long as the users don’t have pay for use (or decide to prioritize) the danger exists for the abuse. So if you have a developer team and everyone just gets to dump tickets on to them and then whine if they don’t get what they want, when they want, I see the analogy as accurate. If people can just treat a resource as though it was suppose to just serve them and the resource is overwhelmed I see that as tragedy of the commons.

There are many ways to manage that problem (some manager deciding the priority for example). Then you may have other problems, but may avoid the tragedy of the commons scenario (in reality this setup is often done, but most don’t accept the prioritizations and just expect the development team to get everything done – which means you don’t avoid the tragedy of the commons problem).
Continue reading

Finding, and Keeping, Good IT People

Finding good IT people, wherever you located globally, is hard. Waiting for the good IT people to apply for positions, is not likely to gain enough good candidates.

To get really good IT people you need to actually manage your current IT staff properly. Then word will get out that your organization is not run by pointy haired bosses (phb) and good IT people will be open to joining. This obviously is not a quick fix. But this practice is the key. This is just respect for people with a eye on the special needs of creative IT people.

If you do this you will also reduce turnover. That doesn’t help in recruiting people, but it solves the underlying problem recruiting is meant to deal with – having staff to do the work. Making your environment tech employee friendly has the benefits mentioned above and will reduce turnover.

Like many issues when examined systemically the most important factors to deal with the recruiting problem are often not directly looking at the problem at hand. Now there are sensible actions to improve the recruiting process. Take a fundamental look at the hiring process and think about some real changes – how about trying people out first, not determining staffing primarily on judgments based on how well then interview. Don’t have silly prerequisites. Why do you need a college degree for an IT job? Or why require specific degrees, like a computer science degree, and exclude others, for example, an online IT degree. Might specific college experience be helpful? Yes. Might someone without it be a great employee? Yes.
Continue reading

Assigning Story Points to Bug Fixes

Agile software development has teams estimate the effort to deliver requests from the product owner. The estimates are done in points (in order to abstract away from hours – as estimates have plenty of variation in how long they will really take). Then the teams capacity (velocity) is determined based on looking at how many points they complete in a “sprint” (a set length, often 2 weeks). Then the product owner can prioritize all of the requests with an understanding of how much effort each is estimated to take and the historical capacity of the development team.

I think it is good to add point estimates to bugs. It may well impact how bugs are prioritized – if it is known to be simple a program manager may say, yes I want these 6 first then… If then know the first 2 are likely to take a bunch of time, they may think, ok, I am not going to get these 4 for awhile… They might just accept that, or may wish to shift more hours to bug fixes this sprint. Or they might say well if it is that big an issue maybe we could do x instead…

In practice I rarely has us estimate emergent bugs we are going to fix in the current sprint, but we do it for bugs that are in the backlog. I sometimes will have us estimate a current bug if I think it is may take significant time – to help determine what we really want to and what the impact may be on the teams output for the sprint. We do not have many emergent significant bugs so it isn’t much of an issue for us.

We do have more difficulty accurately estimating bugs, compared to new stories, but we still provide actionable estimates (they are not perfect, but are usable).

We use agile software development principles at work and they have been a great help in letting us be much more effective than we had been previously. The discussion of priorities and delivery expectations are much improved by such methods I believe. And unrealistic expectations can be reduced. For various reason, without adopting some form of agile/lean… software development methods the common pattern I see is software developers being frustrated by unrealistic expectation of their customers (project managers…) being frustrated by failure to communicate what it is reasonable to expect and status updates. A big part of this is the failure to acknowledge variation (and the related difficulty in estimation). Agile/Kanban… are systems that take the variation into account, and therefore the variation is dealt with as natural instead of leading to bad outcomes for developers and their customers.

Response to Should story points be assigned to a bug fixing story.

Related: Future Directions for Agile ManagementMistake Proofing Deployment of Software ApplicationsChecklists in Software Development

Is Google Failing Too Often?

I think Google is extremely successful, but they do seem to consistently have problems adding to their portfolio. They did a great job with gmail. Android has been very successful. Google Maps is great. They did well building YouTube. Chrome is very nice. Automatic translation is very nice (as is the integration with Chrome).

But so many things just don’t go anywhere. I can’t understand why they can’t take something like Google checkout and make it much more successful (there is money even Google cares about waiting for success in this area). Grand Central was great – Google Voice has not built that the way I would hope. Google has an endless stream of very small companies they buy and then the service dies.

It has been long enough now that I am starting to feel more comfortable saying Google is not doing a good job of creating and building new products. There are a few successes. And having failures isn’t a huge deal – taking risks is wise. But they just seem to be succeeding far to little, especially when you look at the talent and resources they have. Of course, some will say the resources they have is a problem. I really think it is more along the lines I see you mentioning above – they have become too rigid in development. I actually support more standardization than maybe people want (there can be big benefits) but I believe you need to then allow for exceptions. It seems to me Google doesn’t allow enough. It is tempting for managers to want to duplicate the same style that has made adwords and search successful. That might not be the answer for every project though.

They also seem to be driving away to many people with a rigid adherence to proving every little thing. Now I think some of this is a significant part of Google’s success. The trick is not to throw out all such efforts, but to find ways to gain the benefits without crushing innovative people’s will to continue.

I continue to own stock in Google and believe the future is very promising. Google does far more right than they do wrong, but they have room to improve.

Related: Why Google can’t build InstagramObservations of a New GooglerGreat Marissa Mayer Webcast on Google InnovationGoogle: Ten Golden RulesEric Schmidt on Management at Google
Continue reading

Annual Management Blog Review: Software, Manufacturing and Leadership

In my contribution to the 3rd annual management blog roundup I will take a look at 3 blogs: Dennis Stevens, How to implement “Lean Thinking” in a Business and the Three Star Leadership Blog. This year 14 management bloggers contributed to highlight over 40 blogs, be sure to check out all the posts.

photo of Dennis Stevens

Dennis Stevens


Dennis Stevens writes a blog of the same name focused on agile software development principles with a strong focus on Dr. Deming’s ideas and lean thinking.

  • What’s Deming got to do with Agile – “Deming is not about manufacturing. He is about showing management how to create an environment for success. Deming is about culture – and his System of Profound Knowledge creates an environment that is especially effective for knowledge work… In knowledge work, where products are invisible, impact can be difficult to demonstrate. Kanban clearly shows progress and demonstrates the contribution of each person to the delivery of value. Additionally, PDSA provides opportunities for everyone to contribute to improving the quality of the organization’s capabilities.”
  • Kanban Mental Models and Double Loop Learning – “the Kanban cycle supports continuous learning that the team internalizes. Argyris’s model gives us some insight into why Kanban teams are consistently achieving double-loop learning and rapid maturity.”
  • We are Doing QQ All Wrong– “Developers should be using tools that support automated unit testing and only checking in code that passes all their unit tests… Test driven development or test just after development should be ubiquitous – but it is not. Continuous Integration environments that ensure that each check-in results in a valid and testable platform help teams perform integration and build validation.”
  • Shorten and Reduce Variability in Lead Times Using Kanban – ” identify and leverage strategies like reducing waiting, reducing rework, making work ready, defining small size work, and swarming, to improve lead time. Tracking causes of defects and blockages can help make decisions to focus these strategies appropriately. Reducing lead time duration and variability will result in increased predictability, faster feedback, improved flexibility and responsiveness.”
photo of Tracey Richardson

Tracey Richardson

Tracey Richardson writes the How to implement “Lean Thinking” in a Business blog focused on the lean manufacturing and the Toyota Production System.

  • Common Mistakes when we are Problem Solving – “Not utilizing the ‘Power of the gemba’,–or often referred to as “Go see the work/process“.!! I often see teams working together in a room trying to solve the problem by using their experiences, hypothetical guesses, and what their opinion is. I quickly disperse the huddle to “go-see” with their own eyes the current situation.”
  • How many different types of A3’s are there? – “I will briefly describe the 4 different types of A3’s and when to use them based on my experience: Problem Solving A3, Proposal A3, Status Report A3, Strategic Planning A3. All A3’s should follow the PDCA thinking regardless of which type you are working on.”
  • Why is asking “Why” so important? – “It is important to ask why repeatedly when visiting the gemba to determine what is current happening versus what should be happening. In many cases we stop at a symptom to the problem because we are often pressured for results and quickly solving the problem without going past the symptom seems to be the best answer.” [this one is actually from 2009 but I included it anyway – John]

The Achilles’ Heel of Agile

Guest post by Jurgen Appelo

When I wrote this, I was working in a big open office space in the Van Nelle Factory in Rotterdam (see photo). About 100 people work in an office that was the first of its kind in Europe, when it was built in 1929. And more than 80 years later, architecture lovers from all over the world still come to admire it, take pictures, and make drawings. I sometimes waved at them.

photo of open office style at Van Nelle Office
Van Nelle office, reprinted by permission of Stephan Meijer

A big open office space has advantages and disadvantages. Advantages are flexibility and easy communication. The main disadvantage is that it is a shared resource for all who work there. Climate, sound, and light are hard to manage in a space like that, and the optimal configuration for the whole is never optimal for all. But our office manager did the best she could in trying to maximize pleasant working conditions, while maintaining tight rules to keep things under control. A shared open office is not the ideal environment to give people full responsibility over their own working space.

Self-organization is usually promoted in agile software development. But when shared resources are not managed by a central authority, self-organization often results in the Tragedy of the Commons. The name refers to a situation in which multiple self-organizing systems, all acting in their own self-interest, overexploit a shared limited resource, even when they all know it is not in anyone’s interest for this to happen. The impact that humanity has on CO2 levels in the air, trees in the forests, and fish in the sea, is right now the most debated and intensively researched case of the Tragedy of the Commons. Organizations also have shared resources, like budgets, office space, and system administrators. We could see them as the business-equivalent of the air we breathe, the landscape we change, and the fish we eat.

Research indicates that four ingredients (called the four I’s) are needed for sustainability of shared resources [Van Vugt 2009:42]:

  • Institutions [managers] who work on building trusting relationships between competing systems [teams] in order to increase acceptance of common rules;
  • Information that increases understanding of the physical and social environment, in order to reduce uncertainty (because uncertainty results in bias towards self-interest);
  • Identity, or a need for a social “belonging” that encompasses all participants, to improve and broaden one’s sense of community and reduce competition between teams;
  • Incentives that address the need to improve oneself, while punishing overuse and rewarding responsible use.

Research shows that it is imperative that there is some form of management (or governance) to protect these shared resources by working on these four I’s. (I realize that most modern day governments are not setting a good example of how to do that.) In the case of shared resources, whether it concerns money, space, or system administrators, someone outside of the development teams must keep an eye on long-term sustainability instead of short-term gains by individual teams.

The Tragedy of the Commons is the Achilles’ heel of Agile. It takes management to protect that heel, in order to prevent teams from depleting resources, and crippling the organization.

This article is an adaptation from Management 3.0: Leading Agile Developers, Developing Agile Leaders, by Jurgen Appelo. The book will be published by Addison-Wesley, in Mike Cohn’s Signature Series.

Related: Embrace Diversity, Erase Uniformitymanagement 3.0agile software development booksVW Phaeton assembly plant