Information Technology Dark Side

Struggles of a Self-Taught Coder

Information Technology Dark Side header image 2

Metrics That Matter

February 16th, 2009 · No Comments

I’ve always had an allergic reaction to metrics in software development. Over the years, lots of people have commented that they find this strange, given my background in manufacturing/mechanical engineering. I’ve never been able to explain why, but the metrics I’ve seen proposed for software development have never felt as genuinely useful as the type of metrics I’ve seen collected about manufacturing processes.

I’ve recently discovered a very powerful personal health metric’s usefulness: calories. I first started tracking calories with and twitter, then recently switched to Lose It!, an iPhone app recommended to me by a friend. As I’ve tracked calories, I’ve realized how little I knew about nutrition. It was astounding to discover there are more than 10 times the calories in a tablespoon of Hershey’s syrup than there are in a large asparagus stalk. Cheese, sour cream, ice cream, chocolate, soda, and even macaroni and cheese all have a completely different meaning to me than they once did. Counting calories has really changed my perspective and altered my behavior. But counting calories in itself is not the really powerful part – it’s the calorie budget that really makes a difference. I can eat 2,642 calories every day and lose a pound a week. If I eat more than that, I won’t lose weight. If I eat less, I will lose weight more quickly.

You might think that this experience has changed my viewpoint about software development metrics, and you’d be right, in a way. My position hasn’t changed (still haven’t seen a useful one), but I now understand why well enough to articulate the reasoning behind my position. So, here’s my list of what makes a good metric, along with quotes meant to be good counter-examples:

  • A metric must be simple. “A function point is unit of measurement to express the amount of business functionality an information system provides to a user.”
  • A metric must be easily measurable. One person with a simple tool. “Seriously – you want me to track how much time I spend on prod support, project X, and project Y at the minute level?”
  • A metric must be an unambiguous indicator. There should be no debate about the actual value measured itself. “Well, we say it’s 3 function points, but if you look at the underlying code you realize it’s a lot more complicated than it seems.”
  • A metric must be accompanied by boundaries that are clear, indisputable indicators of badness, like my caloric budget. “Yes my project is 25% over budget, but we’re also a bit ahead of schedule in the requirements gathering, so we’ll make it up in the end.”
  • Boundaries must be based on meaningful stuff (my caloric budget is derived from my height and weight and medical tables) as opposed to just pulled out of an orifice. “My boss expects me to find 3 bugs each day, so I bank up bugs and enter them later when I get my quota.”
  • The vast majority of software development metrics I’ve seen in my career violate most, if not all, of these rules.

    Here are some exceptions that I’ve experienced:

  • Hours spent at the office (as opposed to hours working, which is pretty ambiguous in software development – if I think about a coding problem while I shower am I “working”). More than 40 for more than three weeks in a row is bad, bad, bad.
  • Story points completed per iteration. When the same group of people do the estimating and the work over multiple iterations, this becomes a pretty valuable metric and it meets all my criteria. If you don’t know what I’m talking about, read User Stories Applied.
  • I’m starting to think pomodoros might be a good measure of personal productivity over time, but I haven’t been doing it long enough to be sure. It’s certainly a good technique for creating focus.
  • Notice that this is a short list, and there are lots and lots of metrics for software development I’ve seen over the users. Why haven’t they succeeded? Because they almost always break one of my premises of metrics that matter.

    Think you’ve got a metric I would like? Let’s see it then, in the comments.

    If you enjoyed this post, make sure you subscribe to my RSS feed!
    Stumble it!

    Tags: Development

    0 responses so far ↓

    • There are no comments yet...Kick things off by filling out the form below.

    Leave a Comment