Mutation Testing badge with PIT and Stryker Dashboard

Published July 30, 2022.

Written by Maarten Mulders , reading time roughly 3 minutes (455 words).

Earlier, I wrote about integrating PIT with the Stryker Mutator Dashboard. The setup for that was pretty complicated, with some shell scripting that extracted the JSON payload for the report from a JavaScript file. Today, I’m introducing a much simpler approach to that: the Stryker Mutator Dashboard reporter for PIT.

If you’re unfamiliar with the idea of mutation testing, I invite you to read my previous post. It has a high-level explanation of what mutation testing is, and it refers to another article for more details.

As I said, the setup in my previous blog was rather complicated. The shell script I used to extract the JSON was brittle (to say the least) and I found myself copying it to many new projects.

That’s why today, I am introducing a much cleaner and simpler approach: the Stryker Mutator Dashboard reporter for PIT.

Here’s how it works:

It collects mutation testing results from PIT.
It converts the collected mutation testing results into the format that the Stryker Mutator Dashboard expects.
It uploads the converted report to the Stryker Mutator Dashboard.

How to use it?

I’ve tried to make it as simple as possible to use the reporter.

As a prerequisite, make sure there is an environment variable STRYKER_DASHBOARD_API_KEY with the API key that you got when you set up your project.

Now, follow these four steps:

Find the place in your pom.xml where you define the pitest-maven plugin.

Add a dependency to this plugin declaration:

<dependency>
   <groupId>it.mulders.stryker</groupId>
   <artifactId>pit-dashboard-reporter</artifactId>
   <version>0.1.2</version>
</dependency>

Configure PIT to use the new output format:
```
<configuration>
    <outputFormats>
        <format>stryker-dashboard</format>
    </outputFormats>
</configuration>
```
1. Alternatively, if <configuration> is already there, add the <outputFormats>.
2. Similarly, if <outputFormats> is already there, add (or replace) with <format>stryker-dashboard</format>.
Important If you are working on a multi-module Maven project, add the following to the <configuration> block:
```
<pluginConfiguration>
    <stryker.moduleName>${project.artifactId}</stryker.moduleName>
</pluginConfiguration>
```
This will ensure the mutation testing results of the various Maven modules will not mix up in the report. You should not do this if you have a single-module Maven project!
For all the other things, the reporter will autoconfigure itself, given your build runs on any of the following environments:
- GitHub Actions
If your builds run in another environment, please feel free to open an issue.

That’s all - you’re done! Kick off a fresh build, wait a few minutes and enjoy the online report!

Credits

Credit where credit is due: this reporter would not exist without this tremendous work by Wouter Aarts. His reporter contains all the complex mapping from PIT results to mutation-testing-elements’ JSON format.

Post Scriptum

I have published a slightly shorter version of this blog (excluding some historical background) on the Stryker Mutator blog.

Also, the project readme has been extended with instructions for setting up with Gradle.