Metrics and Software Development

Posted on March 27, 2007 by John Hunter

Lean-based Metrics for Agile CM Environments [the broken link was removed] by Brad Appleton, Robert Cowham and Steve Berczuk:

Measure Up! Don’t use metrics to measure individuals in a way that compares their performance to others or isolates the value of their contributions from the rest of the team. The last of the seven principles of Lean software development tells us to “Optimize across the whole.” When measuring value or performance, it is often better to measure at the next level-up. Look at the big-picture because the integrated whole is greater than the sum of its decomposition into parts. Metrics on individuals and subparts often create suboptimization of the whole, unless the metric connects to the “big picture” in a way that emphasizes the success of the whole over any one part.

I agree that measuring individuals is normally not an effective way improve. And “measuring up” can often be valuable. Often a fixation on small process measures can result in improvements that don’t actually improve the end result. But rather than the measure up view, I find looking at outcome measures (to measure overall effectiveness) and process measures (for viewing specific parts of the system “big picture”) the most useful strategy.

The reason for process measures is not to improve those results alone. But those process measures can be selected to measure key processes within the system. Say finding 3 process measures that if we can improve these then this important outcome measure will improve (using PDSA to make sure your prediction is accurate – don’t fall into the trap of focusing on improving that measure even after the data shows it does not result in the desired improvement to the overall results that was predicted).

Also, process measures are helpful in serving as indicators that something is going wrong (or potentially going better than normal). Process measures will change quickly (good ones can be close to real time) thus facilitate immediate remedies and immediate examination of what lead to the problem to aid in avoiding that condition in the future.

Keep them lightweight – start light and see what happens. If you are getting interesting results, consider some deeper investigations into a particular area.

Very good idea. Focus on fewer measures and really pay attention to them. Don’t become buried by all the metrics that can be collected – take the time to see what measures matter and really pay attention to those measures. As you get an excellent handle on those then add some more that seem worthwhile (and don’t hesitate to remove some from your focus as you determine they are not as important).

Related: posts on metrics – Visible Data – Distorting the System (based on misreading data) – Edward Tufte’s new book: Beautiful Evidence – Measuring and Managing Performance in Organizations

This entry was posted in Data, IT, Lean thinking, Management, Management Articles, Software Development, Theory of Constraints and tagged , , . Bookmark the permalink.

5 Responses to Metrics and Software Development

  1. Pingback: Curious Cat Management Improvement Blog » Combinatorial Testing for Software

  2. Steven Leung says:
    May 15, 2009 at 2:44 am

    I was young then, but I remember with some fascination how they would measure performance at Microsoft, by stack ranking them, from first to last in ratings. I’m not sure it fostered improvement as much as it did competition and secrecy, especially given the emphasis on narrow and deep specialization in the hiring process at the time. This measurement style definitely helped shape how software was written.

  3. Pingback: Curious Cat Management Improvement Blog » Predicting Improves Learning

  4. Pingback: Deming and Software Development » Curious Cat Management Improvement Blog

  5. Pingback: Improving Software Development with Automated Tests » Curious Cat Management Improvement Blog

Comments are closed.