Wednesday, March 15, 2006

Software metrics

Lately, there has been a lot of talk at work about the set of bugs filed against our system. Most of that discussion revolves around two metrics: number of open bugs, and the time since the bug was reported.

The powers-that-be at work have decided that the dev team should complete 18 bugs every week, and 2 enhancements. There is no mention of bug severity, scope, or importance to the application. All that matters is the 20 items, and that those 20 items are the oldest items still open in the database.

Larry Osterman has an interesting post about a similar situation. He presents the case from the point of view of measuring tester performance based on number of bugs reported in a given time frame, but I argue that the conclusions are equally valid when talking about the number of bugs closed in a given time frame. Larry's Rules of Software Engineering #2: Measuring Testers by test Metrics Doesn't. Larry's essay got included in Joel Spolsky's book.

Why is it that software project managers refuse to understand that there is more to a bug than it's count? Some bugs are inherently more important than others. Some bugs affect the functionality or profitability of a system, while many bugs are just nit-picky user preferences (or worse, tester preferences) disguised as a "bug".

Here's my real question: If your executive team is focusing on the wrong metric (popular opinion on my team is that they are), whose failure is that? What if your executive team is built of people who have historically had to focus on the number of beans? Who, from the tech team, is responsible for teaching them the way to manage a software project? Or is it their responsibility, as product/project managers, to buy a book, and learn more of the techniques for successfully managing a software project?


Anonymous said...

Don't try to convince them they are wrong, you immediately put them in the position of defending themselves. Take the approach of " I understand what we are trying to accomplish here and I've got a few ideas that I think may help us reach the goal ". You are all on the same team, you all ultimately have the same basic goal.

Cullen Waters said...

You're correct. We are on the same team, and the best solution is to provide an answer to the problems.

My only resonse to that is that I have been presenting my ideas for organization for months. Recently, we held a mostly low-level meeting to discuss this problem, and came up with some really workable ideas. We set about implementing them, and halfway through, got shot down. The key to the plan was to have a two week release schedule, and to have a concrete list of items that would be worked on during that two weeks. The idea was squashed so we could have one week release schedules. One week!