Topic: Agile management
I had an interesting conversation with Al Goerner, a colleague at Valtech, about tracking agile projects. In a presentation, Al recommended tracking separately (a) test coverage, (b) percentage of tests passing, and (c) tests failing that used to pass. I found this strange, since by definition an agile team must have full test coverage (although this may not be 100% LOC coverage literally, due to the way coverage tools work), 100% of tests must pass before the work is deemed "done," and any broken tests must be fixed before the work is deemed "done." I didn't see how the three measures could be separated in a meaningful way, so I asked him what he was trying to achieve.
Al pointed out that many of our engagements are with clients who are at a very early stage of agile adoption. In terms of the agile marathon, they are only now daring to dream of beginning to think about starting to consider the possibility of carefully attempting, in a limited way, some of the practices at maturity level A. Their development teams routinely label code as "done" when it isn't fully covered by tests, when some tests aren't passing, and when previously-passing tests are broken. Their management knows development projects tend to run over cost and time, but they don't know why. The additional metrics are meant to make the causes of cost overruns visible to management and the effects of suboptimal development practices visible to technical staff.
Considering that agile methods are still at an early stage of diffusion — about 12% of companies have experimented with something they label "agile," whatever that means, but on the whole we're stuck at the edge of the chasm — Al is quite right to say most organizations need to see these metrics. Once teams become more comfortable with and more skilled at agile development and they move closer to maturity level B, it should be feasible to reduce the number of categories of measures. When code coverage is normally acceptable and when it is routine for all tests to pass (both new and old tests) before code is deemed "done," then you will know you have achieved a level of maturity that makes some of the additional metrics superfluous. The sooner you can get there, the easier and more valuable your work will become. Until then, it's probably a good idea to make as many issues visible as you can.
