Musings on (Agile) Metrics

Last week I attended a really interesting session at Agile Open NoCal about what are good metrics for Agile teams.  Since I spent a lot of time thinking about this when working with Cardinal Health on their Design Excellence program, here are some thoughts I have.

Metrics are a powerful tool to influence behavior.  They should be applied judiciously and cautiously to any problem since they ALWAYS create unintended and unexpected behavior.  The larger the organization, the more important it is for measurement since that is the way big organizations receive feedback and can adjust their behavior.  However, it is unreasonable to insist all metrics are destructive and irresponsible to expect a large business to fly blind without some sort of data.

I tend to think, most people do not understand how to design good metrics or they create metrics that are easy to gather, i.e. using time sheets or something else similarly foolish, or they pick metrics which are valuable to the data collector, not anyone else.  I don’t claim to be an expert, but I have some ideas which I think are consistent with Lean Thinking and Agile.  There are four guiding principles I can provide:

1. The only metrics that matter are the one tied to customer value or satisfaction.  All the rest are intermediate measurements.
2. The people being measured should have a say in creating the measurement system on which they are judged.
3. Apply the Lean concept of Measure Up when creating your metrics.
4. The best metrics are temporary.  Once the metric has produced the desired outcome, discard it.

Pascal Dennis in his book, Getting the Right Things Done (not my favorite book – long on story, short on specifics), gives some great guidance on the first principle when flushing out the concept of True North. True North is an emotional description of the strategic and philosophical goals of the organization tied to real-life, customer-focused metrics.  Not the kinds of metrics middle managers care about or software teams, but the types of metrics that show up in reports to shareholders and are on the minds of the executives and senior leaders.  These are the metrics EVERYONE in the organization needs to be focused on and thinking about every day.

In my experience, the people doing the work know what metrics are meaningful to them and which have no impact on their day-to-day activities.  To create a powerful measurement system that gives accurate, real-time data, engage the people doing the work in creating this system.  In addition, the metrics they come up with, are the metrics on which they are evaluated.  The common objections one runs across when developing this idea with middle manager are “what if the workers don’t come up with anything?” or “what if they are not sophisticated enough?”  I feel if you explain to the workers what is the True North, the main business metrics, they will come up with meaningful metrics that tells them if their work is contributing to the objectives of the organization.  The people who work in the company want the business to succeed (and if you feel otherwise, you have a different, more serious problem).