An Exploration of Error Quotient in Multiple Contexts

Data-driven metrics based on students' progress, code or behaviour are touted as a pathway to revolutionizing education. Even simple heuristics, like the recurrence of the same error in subsequent compilations, have the potential to identify students in difficulty, leading to targeted interventions, alterations in teaching, and improved counseling. However, although more complex and effective metrics have been explored in the literature, they have largely been developed and validated in narrow contexts.

In this work, we investigate the application of the Error Quotient (EQ), a metric based on debugging of syntax errors, in a varied set of contexts, including courses taught in C, Java, and Python and courses that use sets of short, online exercises. We perform an exploration of the EQ parameters to determine the EQ algorithm's sensitivity to its context and identify factors in our contexts that have an impact on the process students use to generate code.