(\_ -> Phil Varner)

Several months ago, I began a concerted effort at work to get our code coverage numbers up. This was prompted by an upper management target of 85% code coverage by a certain date, which I initially saw as unrealistic within any timeframe. I hadn't done much work with code coverage, but I did know the primary drawback, in that most tools simply show that code was executed and not that all paths through the code were executed (branch coverage). Any simple metric has the potential to be abused by naive management, since it's easier to measure code coverage than measure if the code is actually being tested correctly and fulfills the desired usecases (assuming they even exist!).

Several months later, now at 85%, I have a more positive and specific view:

Coverage is to testing as compiling is to coding.

That is, it doesn't ensure that your testing is complete, or correct, or anything, but it does make sure that nothing is completely wrong. If code is never even executed, you have zero assurance that it is correct, just as if code that won't compile has zero assurance of being correct. That doesn't mean it is correct, just that it has a non-zero probability of being correct. The realities of software development allow you to only increase the probably of correctness, so this is one more tool to do this.

I found large blocks of code that weren't being run at all, for various reasons. There were a few methods that intended to override superclass methods (and weren't annotated with @Overrides because the code was originally 1.4 based), but had subtle name or signature typos. Some code had subtle logic errors in branches which prevented one way from executing.

Code coverage metrics were particularly useful in the case I was in, where I had inherited a large amount of complex code from another developer who hadn't provided the most thorough set of tests. I could easily see what code wasn't being executed, and then devise test cases to cover these. One has to be very careful when doing this, since you only get one chance to test that section of code correctly, since after the first test which covers the code, you no longer have the obvious warning of uncovered code. This presents the developer with a moral hazard, since they can write the simple test to get their code coverage numbers up or they can write exhaustive tests which correctly test the code and contribute to genuine code quality. You only get one chance to do the right thing.