SonarQube

 

How to Measure Code Coverage Using SonarQube and Jacoco

By Tom GregoryPosted on December 13, 2019

Last Updated on November 7, 2022

Code coverage is a metric that teams use to measure the quality of their tests, and it represents the percentage of production code that has been tested. Discover how to apply the Gradle Jacoco plugin to your project and run a SonarQube scan to generate a code coverage report.

Contents

1. Overview

2. A worked example

3. Setting up a SonarQube instance

4. Using the Jacoco Gradle plugin

5. Using the SonarQube Gradle plugin

6. Viewing the SonarQube report details

7. SonarQube 9

8. Conclusion

9. Resources

UPDATED in November 2021 to reflect SonarQube LTS version switching to 8.9.

1. Overview

SonarQube is a tool which aims to improve the quality of your code using static analysis techniques to report:

• code coverage
• bugs
• code smells
• security vulnerabilities

The SonarQube server is a standalone service which allows you to browse reports from all the different projects which have been scanned. To scan a specific codebase you run the SonarQube scanner. This is a local process that analyses your code then sends reports to the SonarQube server.

Code coverage

The code coverage metric has to be computed outside of SonarQube using a different tool. The result is then ingested into SonarQube and shown within its web interface.

The tool we’ll look at today to calculate code coverage for a Java project is called Jacoco. Jacoco analyses the code and generates an XML report, which is later ingested by SonarQube.

2. A worked example

Let’s run through an example of exactly how Jacoco and SonarQube work together to calculate code coverage. To do this we’ll use the sonarqube-jacoco-code-coverage GitHub project, which you can clone to see it working with your own eyes.

We’ll set up:

1. an instance of SonarQube running in Docker
2. a Java project with a class and some unit tests
3. a Gradle build which generates a Jacoco code coverage report then runs the SonarQube scanner against the Java project

 Gradle is a popular build tool for building Java applications. It can also generate a Jacoco code coverage report and run the SonarQube scanner, as you’ll see later in the article. Learn more with this beginner tutorial.

The project has 2 classes:

1. A MathService class which has 2 methods, multiply and subtract
2. A MathServiceTest class which only tests 1 out of the 2 methods in MathService

So we’re hoping that SonarQube will highlight the fact that we’re missing a test here i.e. only one of the methods in MathService has been tested. Any guesses for what percentage code coverage SonarQube will report in this case? 

 Prerequisites: to follow along with this example, you’ll need installed locally both Docker and Java 8+. It’s been tested with Java 8, 11, and 17.

3. Setting up a SonarQube instance

An official Docker image exists for SonarQube, making this really easy to get up and running using Docker Compose. Just add the following docker-compose.yml file to your project:

version: "3"

services:

sonarqube:

image: sonarqube:lts

ports:

- 9000:9000

environment:

- SONAR_FORCEAUTHENTICATION=false

• uses the lts (long term support) version of the SonarQube Docker image, which is currently SonarQube version 8.9.
• SonarQube is configured to start on port 9000
• SonarQube authentication is disabled to simplify this demo. Always use authentication in a production environment.

To run Docker Compose from Gradle, just add this plugin to your build.gradle:

plugins {

id 'com.avast.gradle.docker-compose' version '0.14.3'

// any other plugins

}

 if you’re following along with the example GitHub repository rather than applying this to your own project, the previous and following steps are already configured for you.

Now we can run ./gradlew composeUp to start the SonarQube Docker container through Gradle:

$ ./gradlew composeUp

> Task :composeUp

sonarqube uses an image, skipping

...

BUILD SUCCESSFUL in 19s

1 actionable task: 1 executed

That was successful, but we can double check everything is OK by seeing what Docker containers are running with docker ps:

$ docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

35459fcff334 sonarqube:lts "./bin/run.sh" About a minute ago Up About a minute 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp 39b4a0caef82cbb00dd981c87ae00

c20_sonarqube-jacoco-code-coverage__sonarqube_1

This shows that SonarQube is running on localhost:9000. It might take a minute for SonarQube to start up, but eventually within a browser you’ll be able to reach the main SonarQube dashboard.

This is correctly reporting we currently have 0 projects analysed. Let’s fix that! 

4. Using the Jacoco Gradle plugin

Before we get onto actually scanning our code with SonarQube, let’s set up the Jacoco Gradle plugin. This will generate the test coverage statistics for our Java code. Just add the following plugin definition and configuration to build.gradle:

plugins {

id 'jacoco'

// any other plugins

}

jacocoTestReport {

reports {

xml.enabled true

}

}

test.finalizedBy jacocoTestReport

• we apply the jacoco plugin, a core Gradle plugin
• we configure the plugin to output test reports in the XML format required by SonarQube
• we use finalizedBy to ensure that the test report is always generated after tests are run

Now let’s run ./gradlew test. Notice we have a new file reports/jacoco/test/jacocoTestReport.xml created in the build directory.

This contains the code coverage information that SonarQube will pick up during its scan.

5. Using the SonarQube Gradle plugin

Now that we’ve got our test code coverage data being generated by Jacoco, it’s time to hook all this up by running a SonarQube scan. This will report on the code coverage as well as run a full scan of our code. To do this we’ll use the SonarQube Gradle plugin which adds the sonarqube task to our build.

We can include it in our build.gradle like this:

plugins {

id 'org.sonarqube' version '3.3'

// any other plugins

}

We also need to include a configuration to tell the SonarQube scanner where to find the SonarQube server:

sonarqube {

properties {

property 'sonar.host.url', 'http://localhost:9000'

}

}

Lastly, to ensure the Jacoco test report is always created when we run the sonarqube task let’s setup the following dependsOn relationship:

tasks.named('sonarqube').configure {

dependsOn test

}

Now we just need to run the sonarqube Gradle task to run a scan, with ./gradlew sonarqube.

$ ./gradlew sonarqube

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 10s

5 actionable tasks: 1 executed, 4 up-to-date

Success!

6. Viewing the SonarQube report details

We can head back to SonarQube at localhost:9000/projects to see the test code coverage report.

We can see a reported code coverage of 66.7%. Awesome! Click on the sonarqube-jacoco-code-coverage link and we’ll try to drill into exactly how this was calculated.

This is a more detailed view of the report. Click on the 66.7% link.

We now see information about what class has been analysed, in this case the MathService. Click on the link to see even more details:

We can now see the class itself, where green highlights code that is properly tested and red code that isn’t. Let’s zoom in a bit:

We can see that SonarQube is telling us that:

• the class MathService is covered by tests (green mark)
• the multiply method is covered by tests (green mark)
• the subtract method is not covered by tests (red mark)

That makes 2 out of 3, hence the 66.7% being reported by SonarQube. So there’s definitely room for improvement!

7. SonarQube 9

The above example uses the LTS version of SonarQube, currently 8.9. The same steps can be used to run against SonarQube 9.

You can view the SonarQube version at the bottom of any page on the web UI.

8. Conclusion

You’ve seen that it’s really easy to setup code coverage reporting in a Gradle project using Jacoco and SonarQube. Try it out on your own project to see how you measure up.

Note that this isn’t the only metric you should use to measure your test quality, but it can be a helpful indicator. Code may have a high code coverage percentage, but it might be brittle and difficult to maintain.

To continue learning, check out these super-relevant articles:

• did you think about integrating SonarQube into your continuous integration process? Learn how to set this up in Jenkins in the article Add SonarQube quality gates to your Jenkins build pipeline.
• do you want to include coverage data from integration tests? Find out how in Integration Test Code Coverage with SonarQube and Jacoco

Get the Build Boss newsletter

Found this website helpful? Subscribe for updates.

 All of my latest articles each week
 Access to video tutorials
 Exclusive tips and offers not found on my website

9. Resources

GITHUB REPOSITORY
Follow along with this article by checking out the accompanying GitHub repository. This uses the LTS version of SonarQube (currently version 8.9).

GRADLE
Jacoco Plugin docs
SonarQube Plugin docs
Tutorial for complete beginners

SONARQUBE
General documentation

VIDEO
If you prefer to learn in video format, check out this accompanying video on the Tom Gregory Tech YouTube channel.