Grammatical Error Correction for Sentence-level Assessment in Language Learning

Tutkimustuotos: Artikkeli kirjassa/raportissa/konferenssijulkaisussaKonferenssiartikkeliTieteellinenvertaisarvioitu

Abstrakti

The paper presents experiments on using a Grammatical Error Correction (GEC) model to assess the correctness of answers that language learners give to grammar exercises. We hypothesize that a GEC model corrects only errors and leaves correct answers unchanged. We perform a test on assessing learner answers in a real but constrained language-learning setup: the learners answer only fill-in-the-blank and multiple-choice exercises. For this purpose, we use ReLCo, a publicly available manually annotated learner dataset in Russian (Katinskaia et al., 2022). In this experiment, we fine-tune a large-scale T5 language model for the GEC task and estimate its performance on the RULEC-GEC dataset (Rozovskaya et al., 2019) to compare with top-performing models. We also release a manually checked, cleaner version of the RULEC-GEC test set. Our analysis shows that the GEC model performs reasonably well in detecting erroneous answers to grammar exercises and potentially can be used for best-performing error types in a real learning setup. However, it struggles to assess answers which were tagged by human annotators as alternative-correct using the aforementioned hypothesis. This is in large part due to a still low recall in correcting errors, and the fact that the GEC model may modify even correct words---it may generate plausible alternatives, which are hard to evaluate against the gold-standard reference.
Alkuperäiskielienglanti
OtsikkoProceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications
ToimittajatEkaterina Kochmar, Jill Burstein, Andrea Horbach, et al.
Sivumäärä15
JulkaisupaikkaStroudsburg
KustantajaThe Association for Computational Linguistics
Julkaisupäiväheinäk. 2023
Sivut488–502
ISBN (elektroninen)978-1-959429-80-7
TilaJulkaistu - heinäk. 2023
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaWorkshop on Innovative Use of NLP for Building Educational Applications - Toronto, Kanada
Kesto: 13 heinäk. 202313 heinäk. 2023
Konferenssinumero: 18
https://sig-edu.org/bea/2023

Tieteenalat

  • 6121 Kielitieteet

Siteeraa tätä