Can Students Review Their Peers?: Comparison of Peer and Instructor Reviews

Tutkimustuotos: Artikkeli kirjassa/raportissa/konferenssijulkaisussaKonferenssiartikkeliTieteellinenvertaisarvioitu


Having students peer review each other's exercises is a common task in modern computing classrooms. In large classes, peer review might even partly replace traditional instructor-led review -- and prior work has found some indications that the quality of peer reviews can be close to that of instructor reviews. In this work, we explore the difference between instructor and peer reviews of student-created programming exercises. One task in an introductory programming course was to have students design their own programming exercises -- including an exercise description, model solution, and test cases -- which were then reviewed by peers. After the course, we had two instructors review the same student-created exercises. We compare the scores given by the instructors and the students to analyze potential differences. Our results suggest that agreement between instructors and students as measured by inter-rater reliability is low, although differences between instructor and student review score distributions are not statistically significant. Additionally, instructors have more fluctuation in their reviews compared to students. Due to the rising popularity of peer reviews, more research is needed to examine to what extent they could complement traditional instructor-led review of exercises.
OtsikkoITiCSE '22: Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1
JulkaisupaikkaNew York
Julkaisupäiväheinäk. 2022
ISBN (elektroninen)978-1-4503-9201-3
DOI - pysyväislinkit
TilaJulkaistu - heinäk. 2022
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaITiCSE '22: Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education - Dublin, Irlanti
Kesto: 11 heinäk. 202213 heinäk. 2022
Konferenssinumero: 27


  • 113 Tietojenkäsittely- ja informaatiotieteet
  • 516 Kasvatustieteet

Siteeraa tätä