Annotating Errors in English Learners’ Written Language Production: Advancing Automated Written Feedback Systems
Coyne, Steven Galvan-Sosa, Diana Spring, Ryan Guerraoui, Camelia Zock, Michael Sakaguchi, Keisuke Inui, Kentaro
Coyne, Steven
Galvan-Sosa, Diana
Spring, Ryan
Guerraoui, Camelia
Zock, Michael
Sakaguchi, Keisuke
Inui, Kentaro
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Recent advances in natural language processing (NLP) have contributed to the development of automated writing evaluation (AWE) systems that can correct grammatical errors. However, while these systems are effective at improving text, they are not optimally designed for language learning. They favor direct revisions, often with a click-to-fix functionality that can be applied without considering the reason for the correction. Meanwhile, depending on the error type, learners may benefit most from simple explanations and strategically indirect hints, especially on generalizable grammatical rules. To support the generation of such feedback, we introduce an annotation framework that models each error’s error type and generalizability. For error type classification, we introduce a typology focused on inferring learners’ knowledge gaps by connecting their errors to specific grammatical patterns. We collect a dataset of annotated learner errors and corresponding human-written feedback comments, each labeled as a direct correction or hint. With this data, we evaluate keyword-guided, keyword-free, and template-guided methods of generating feedback using large language models (LLMs). Human teachers examined each system’s outputs, assessing them on grounds including relevance, factuality, and comprehensibility. We report on the development of the dataset and the performance of the systems investigated.
Citation
S. Coyne et al., “Annotating Errors in English Learners’ Written Language Production: Advancing Automated Written Feedback Systems,” Lecture Notes in Computer Science, vol. 15880 LNAI, pp. 292–306, 2025, doi: 10.1007/978-3-031-98459-4_21/FIGURES/6
Source
Proceedings of the International Conference on Artificial Intelligence in Education
Conference
26th International Conference, AIED 2025
Keywords
Subjects
Source
26th International Conference, AIED 2025
Publisher
Springer Nature