Java Code Coverage: Filtering out the noise

Code Coverage is a measure of how much of your code has been executed. This is usually measured while running unit tests as a way of tracking how much production code is being exercised. It is important to note that high code coverage doesn't give any guarantee that the behaviour of the covered code is actually being asserted on by your unit tests but it is a useful indicative measure.

However, there are a couple of serious drawbacks with most simple code coverage tools.

Implicit Code

Java source code is compiled to bytecode before being executed. The most common mechanism for tracking Java code coverage is to instrument this bytecode. The problem with this approach is that it can lead to reporting of coverage (or lack of) for elements which are not present in the actual Java source code. This is frustrating as it either lowers the overall code coverage figure or forces a developer to write pointless extra tests.

Examples:

• Classes without an explicit constructor have an implicit no-args constructor.
• Enum classes have implicit values() and valueOf(String) methods.
• Synchronized(object) {} blocks are implemented in bytecode by duplicating the instructions for releasing the lock inside and outside of a catch block. This would naively require an exception to be thrown from within every synchronized block to achieve full coverage.
• All finally {} blocks are implemented in bytecode by duplicating blocks of instructions possibly several times. This would naively require every branch within the finally block to be exercised for every duplicate.

Uncovered Code

It is commonly acknowledged that 100% code coverage is an expensive goal to attempt to reach. This is usually because getting the final 5-10% code coverage involves writing pointless extra tests that add little real value and can require production code to be distorted to support the pointless extra tests.

JaCoCo

JaCoCo is an open source Java Code Coverage tool which is in active development. It is a mature project and since late 2011 has been the library behind the popular Eclipse plugin EclEmma.

Over the last few months I have been working to add some key features to this library to address the concerns discussed above.

I have updated JaCoCo to automatically filter out coverage of all of the following:

• Implicit no-args constructors.
• Implicit enum methods.
• Synchronization exit handling.
• Finally block code duplication.

I have also added support for source directives that allow blocks of code to be explicitly excluded from code coverage reports. This addresses the issue of uncovered code and allows developers to explicitly mark blocks of code which are deliberately not being covered by unit tests.

All of these changes are ready to try out now by downloading the release of my JaCoCo fork from the link below.

I am working with the JaCoCo developers to get my changes accepted back into the core JaCoCo project.

More Information

http://mchr3k.github.com/jacoco/