Metrics and Software Development

Lean-based Metrics for Agile CM Environments by by Brad Appleton, Robert Cowham and Steve Berczuk:

Measure Up! Don’t use metrics to measure individuals in a way that compares their performance to others or isolates the value of their contributions from the rest of the team. The last of the seven principles of Lean software development tells us to “Optimize across the whole.” When measuring value or performance, it is often better to measure at the next level-up. Look at the big-picture because the integrated whole is greater than the sum of its decomposition into parts. Metrics on individuals and subparts often create suboptimization of the whole, unless the metric connects to the “big picture” in a way that emphasizes the success of the whole over any one part.

I agree measuring individuals is normally not an effective way improve. And “measuring up” can often be valuable. Often a fixation on small process measures can result in improvements that don’t actually improve the end result. But rather than the measure up view, I find looking at outcome measures (to measure overall effectiveness) and process measures (for viewing specific parts of the system “big picture”) the most useful strategy.

The reason for process measures is not to improve those results alone. But those process measures can be selected to measure key processes within the system. Say finding 3 process measures that if we can improve these then this important outcome measure will improve (using PDSA to make sure your prediction is accurate – don’t fall into the trap of focusing on improving that measure even after the data shows it does not result in the desired improvement to the overall results that was predicted).

Also, process measures are helpful in serving as indicators that something is going wrong (or potentially going better than normal). Process measures will change quickly (good ones can be close to real time) thus facilitate immediate remedies and immediate examination of what lead to the problem to aid in avoiding that condition in the future.

Keep them lightweight – start light and see what happens. If you are getting interesting results, consider some deeper investigations into a particular area.

Very good idea. Focus on fewer measures and really pay attention to them. Don’t become buried by all the metrics that can be collected – take the time to see what measures matter and really pay attention to those measures. As you get an excellent handle on those then add some more that seem worthwhile (and don’t hesitate to remove some from your focus as you determine they are not as important).

Related: posts on metricsVisible DataDistorting the System (based on misreading data)Edward Tufte’s new book: Beautiful EvidenceMeasuring and Managing Performance in Organizations

5 thoughts on “Metrics and Software Development

  1. Pingback: Curious Cat Management Improvement Blog » Combinatorial Testing for Software

  2. I was young then, but I remember with some fascination how they would measure performance at Microsoft, by stack ranking them, from first to last in ratings. I’m not sure it fostered improvement as much as it did competition and secrecy, especially given the emphasis on narrow and deep specialization in the hiring process at the time. This measurement style definitely helped shape how software was written.

    Reply
  3. Pingback: Curious Cat Management Improvement Blog » Predicting Improves Learning

  4. Pingback: Deming and Software Development » Curious Cat Management Improvement Blog

  5. Pingback: Improving Software Development with Automated Tests » Curious Cat Management Improvement Blog

Leave a Reply

Your email address will not be published. Required fields are marked *