Model-Driven Engineering (MDE) has emerged as a promising approach for software development, emphasizing the generation of code from conceptual models. While this approach ensures code correctness with respect to the model, it’s crucial to validate the model against system requirements. Model testing, a technique that allows users to try out real-world scenarios on the model, plays a pivotal role in this validation process.
Students often grapple with conceptual modeling due to the multiplicity of potential solutions. Previous research has demonstrated the effectiveness of model testing tools in enhancing student understanding of model behavior and structure. However, the question remains: Are there specific testing behaviors that correlate with high performance in MDE courses?
We collected user logs of students interacting with a model-testing tool. Process mining techniques were employed to analyze the log data and create a process model representing the students’ testing behavior. The process model was analyzed to identify patterns and differences in testing behavior between high-performing, average-performing, and low-performing students.
The analysis revealed distinct behavioral patterns among students of varying performance levels. High-performing students exhibited the following behaviors:
- Frequent experimentation: They actively explored different testing scenarios on the model.
- Targeted testing: They focused on testing specific aspects of the system to validate their understanding.
- Effective feedback utilization: They leveraged feedback from the testing tool to refine their models.
In contrast, low-performing students tended to:
- Limited experimentation: They were less likely to try different testing scenarios.
- Random testing: Their testing efforts often lacked a clear purpose or focus.
- Difficulty interpreting feedback: They struggled to understand and apply feedback on their testing efforts given by the testing tool.
Read our full paper to also get recommendations for the teaching of conceptual modelling: Soon avaliable
