Agile Control Charts
I stumbled across Agile Shrugged today and read a post on how the author experienced a lot of variation in their team velocity. The author did not seem too concerned with all the “sawtooth” behavior of velocity chart because they felt the velocity graph was a “fairly accurate measure of our teams velocity”. Let’s see if this qualitative assessment matches up with statistical process control, one of the Six Sigma tools.
In Six Sigma, a common tool Green Belts and Black Belts use to monitor their processes are control charts. The control chart is an objective measurement tool to determine if the process under inspection can reliably produce what is expected of it in the future. By examining the patterns of the data on the control chart we can understand if the process is “in control” or “out of control”. If the process is “out of control”, then we need to inspect the process, discover the cause of the variation, “mod” the process and monitor the changes with…a control chart.
The diagram below is the velocity data from the team. You can see the “sawtooth”, or oscillation, in velocity over the course of ten iterations. The team does seem to have improved their velocity over time, but we still have an order of magnitude difference in the completed output of the team from week-to-week. Is this significant? Should we care about the oscillation?
To make a control chart, we need to run some basic statistics on the data: compute the mean, standard deviation (σ) and determine the upper and lower spec limits (mean ± 3σ). Here are the results for this data set:
mean: 6.6 ideal days upper spec limit (USL): 24 ideal days σ: 5.8 ideal days lower spec limit (LSL): -10.8 ideal days
Next, we will plot the data on a control chart showing the mean (green line), the USL (red line), 1σ and 2σ (purple lines). I have eliminated the LSL because a negative velocity is meaningless.
Now here is the interesting part: looking at the patterns. Using the Nelson Rules, we can examine the patterns in the control chart to see if the process is “in control” or “out of control”. Here are the rules that I think are relevant for this chart (please review all the rules if you are interested in learning more):
- One point is more than 3 standard deviations from the mean. Nope
- Fourteen (or more) points in a row alternate in direction, increasing then decreasing. Only 8 points so far; maybe something to watch out for.
- Four (or five) out of five points in a row are more than 1 standard deviation from the mean in the same direction. Only 2 out of 5 data points.
What do we find by creating and examining the control chart? Statistical confirmation that our qualitative opinion that our planning process is indeed fine. Great! Keep up the good work!
