Be Careful What You Measure

Paul Graham's recent post Startups in 13 Sentences got my attention with #7, "You make what you measure." If you want to improve some aspect of your business (or software, or whatever it is you do), measure it. Keep daily track of it and you'll start to notice what sorts of things cause it to go up and down. Do more of those things that make it better.

What I found particular insightful was the corollary, "Be careful what you measure." It made me immediately think of the following quote from Microsoft founder Bill Gates.

Measuring programming progress by lines of code is like measuring aircraft building progress by weight.

If you give programmers incentive by measuring the number of lines of code that they write, then that's what they'll concentrate on. It doesn't really even matter if you want more lines of code or fewer, you'll get exactly what you incentivize. Most likely at the expense of quality.

Eli Goldratt explores this idea in depth in his book Critical Chain. The relevant example in the book is a steel processing plant, but can be effectively adapted to any business. In the book, the new managers of the plant find that employees are cutting corners on quality and efficiency in order to meet their performance incentives. The managers turned the company from the brink of bankruptcy into a profitable company simply by removing the incentives. The lesson? No measurement at all is better than making measurements that cause you to focus on the wrong thing.