Software testing is a meaningful way to increase confidence in software quality, but its effectiveness depends on the test suite’s quality. One way to measure test suite quality is through mutation analysis. However, introducing mutation analysis and testing into software engineering curricula presents some challenges. One of the challenges is related to the number of mutation testing tools available.
In a hands-on project, assigning different mutation testing tools to students may generate inequality between them because the tools can be more or less difficult to use and install. To overcome this problem, we are developing an environment/infrastructure that integrates different mutation tools, allowing students to interact with these different tools in the same manner. Thus, we eliminate differences in tool usage while allowing students to realise that tools work differently, for example, generating different mutants for the same piece of code.
The prototype of the infrastructure is available and was already used in a software testing course. Students wrote test cases to kill mutants generated by different mutation testing tools using the same infrastructure with PIT, Major, and Judy integrated.
This first use of the infrastructure allows us to validate and improve it according to the feedback received from the students.
This infrastructure is being improved and extended to help the teachers with the final assessment. This assessment module automatically runs all student test cases and gathers metrics such as mutation scores. It produces a final matrix comparing data provided by students and data obtained by running their test cases. This information is helpful for the final assessment.
The first prototype of this infrastructure is available at https://github.com/PedroTav/defects4j
