Be Careful What You Measure by Mike Wroblewski:

Excellent points. Behavior can be changed by what is measured. The problems with arbitrary numerical targets (to take one measurement related example) is not that attempts to achieve those targets won’t have an affect. They very well may have an affect. However they may not have the desired result. When focused on improving a number (which can happen when focused on measures - especially as the focus on those measures is tied to bonuses, favorable treatment…) the focus is not necessarily on on improving the system. Often distorting the system is the result.

Measures need to be used with a conscience effort to remember the data is merely a proxy to quantify the results (not the end themselves). Taking care in choosing the measures is one necessary step to assure the best improvement results. One strategy is to include some measures that are outcome measures. Often those measures are difficult to pin to specific process improvements tightly so you will also want to include specific process measures. The outcome measures help make sure you maintain a focus on the important system level results. Process measures will help you test and improve processes (as well as monitor and react, when necessary to ongoing processes).

Often improving the process measures can be mistaken for the aim. Care needs to be taken to underscore the role of process measures (process management). Also measures should be re-examined periodically to determine if they are still the correct measures. Systems with people are heavily influenced by what is measured. People will often react to what is measured and make adjustments to how the work is done to make the numbers better. The danger is that those attempts to make the measures look better can actually harm the overall system (when poor measures are used).

Related: Targets Distorting the System - Understanding Data - Operational Definitions and Data Collection - Dangers of Forgetting the Proxy Nature of Data