Loading...
Automatic Feedback Generation for Japanese Reading Comprehension Short-Answer Questions Using Discourse-Structure-based Answer Diagnostic Graphs
Furuhashi, Momoka Funayama, Hiroaki Matsubayashi, Yuichiroh Shimmei, Machi Isobe, Yoriko Ishii, Yutaka Inui, Kentaro
Furuhashi, Momoka
Funayama, Hiroaki
Matsubayashi, Yuichiroh
Shimmei, Machi
Isobe, Yoriko
Ishii, Yutaka
Inui, Kentaro
Files
Loading...
33_240.pdf
Adobe PDF, 3.64 MB
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Journal article
Date
License
http://creativecommons.org/licenses/by/4.0/
Language
Japanese
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Short-Answer Reading Comprehension (SARC) questions are widely used in reading education. However, providing feedback on student responses requires substantial time and effort due to the wide variety of incorrect responses. To address this, we propose an automatic feedback-generation method for questions that require an understanding of the logical structure of a given prompt text. We introduce an Answer Diagnostic Graph (ADG) that represents the text's logical structure as a graph: nodes are textual elements and edges represent logical relationships. Feedback templates are linked to edges to explain how the parts of the prompt text are logically connected. To generate feedback, we formulated a classification task that identifies the reference node, from the part of the text that the student is likely to refer to based on the ADG. The experiments included technical and human evaluations by four experts. Technical evaluation showed high accuracy of reference-node prediction for most questions, suggesting that it influenced human judgment. Human evaluations compared feedback from our method and large language model (LLM) across four criteria: reference indication, logical structure, difference from model answers, and overall satisfaction. Our proposed method outperformed the LLM on all criteria, with statistically significant differences (p < .001) for three of the four evaluators, demonstrating the effectiveness of the approach.
Citation
M. Furuhashi, H. Funayama, Y. Matsubayashi, M. Shimmei, Y. Isobe, Y. Ishii , et al., "Automatic Feedback Generation for Japanese Reading Comprehension Short-Answer Questions Using Discourse-Structure-based Answer Diagnostic Graphs" Journal of Natural Language Processing, vol. 33, no. 1, pp. 240-282, 2026, https://doi.org/10.5715/jnlp.33.240.
Source
Journal of Natural Language Processing
Conference
Keywords
46 Information and Computing Sciences, 4605 Data Management and Data Science, 4 Quality Education
Subjects
Source
Publisher
Association for Natural Language Processing