Metrics are valuable when they are actionable. Think about what will be done if certain results are shown by the data. If you can’t think of actions you would take, it may be that metric is not worth tracking.
Metrics should be operationally defined so that the data is collected properly. Without operationally definitions data collected by more than one person will often include measurement error (in this case, the resulting data showing the results of different people measuring different things but calling the result the same thing).
And without operational definitions those using the resulting data may well mis-interpret what it is saying. Often data is presented without an operational definition and people think the data is saying something that it is not. I find most often when people say statistics lie it is really that they made an incorrect assumption about what the data said – which most often was because they didn’t understand the operational definition of the data. Data can’t lie. People can. And people can intentionally mislead with data. But far more often people unintentionally mislead with data that is misunderstood (often this is due to failure to operationally define the data).
In response to: Metrics Manifesto: Raising the Standard for Metrics
Related: Outcome Measures – Evidence-based Management – Metrics and Software Development – Distorting the System (due to misunderstanding metrics) – Manage what you can’t measure
I think that your post is spot on!
I like folks to ask the simple question of, “So what?” As in, what does this metric mean, what does it mean to me, is it actionable, etc.? Any/all metrics that are reflected say in a tiered performance metric board should have a short profile (typically right behind it or to the side of the board in a metric matrix) that explains the name of the metric, the source data, the calculation(s) (if any), who compiles or is responsible for the metric, the frequency of reporting and the implications (the so what).
There are other considerations relative to the simplicity of the presentation, inclusion of targets, trending, etc. But, perhaps for another time…
Pingback: Indirect Improvement » Curious Cat Management Improvement Blog
Pingback: Make Your Bounce Rate More Meaningful | Coding and Server Syntax Examples
Pingback: Data are not taken for museum purposes; they are taken as a basis for doing something. « The W. Edwards Deming Institute Blog