How do Mutation Testing Tools Differ?

It is not always easy to understand how different mutation testing tools work just by using them.

This article presents a research work with two main goals:

(1) Develop a framework to compare mutation testing tools in a systematic and fair way. The framework allows comparing mutation tools, considering five dimensions: Tool version; Deployment; Mutation process; User-centric features; and Mutation operators. Based on this framework, the researchers compare 8 state-of-the-art Java mutation tools to highlight their similarities and differences.

(2) To understand, based on the results of a questionnaire to researchers, practitioners, and educators, which are the key features these three different stakeholders consider when selecting a mutation testing tool for their work.

With the result of this work, it is possible to understand the state of the art to make informed decisions about which features to implement, improve, or extend in the future when developing new mutation testing tools based on their target audience. Also, to perform empirical studies it is useful to know how the tools work internally to make a fair comparison.

Read more about this study in the following paper: Domenico Amalfitano, Ana C. R. Paiva, Alexis Inquel, Luís Pinto, Anna Rita Fasolino, and René Just. 2022. How do Java mutation tools differ? Commun. ACM 65, 12 (December 2022), 74–89. https://doi.org/10.1145/3526099