Identification of Multiple Logical Interpretations in Counter-Arguments
Wang, Wenzhi Reisert, Paul Naito, Shoichi Inoue, Naoya Shimmei, Machi Pothong, Surawat Choi, Jungmin Inui, Kentaro
Wang, Wenzhi
Reisert, Paul
Naito, Shoichi
Inoue, Naoya
Shimmei, Machi
Pothong, Surawat
Choi, Jungmin
Inui, Kentaro
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Counter-arguments (CAs) are a good means to improve the critical-thinking skills of learners, especially given that one has to thoroughly consider the logic of initial arguments (IA) when composing their CA. Although several tasks have been created for identifying the logical structure of CAs, no prior work has focused on capturing multiple interpretations of logical structures due to their complexity. In this work, we create CALSA+, a dataset consisting of 134 CAs annotated with 13 logical predicate questions. CALSA+ contains 1,742 instances annotated by 3 expert annotators (5,226 total annotations) with good agreement (Krippendorff 𝛼=0.46). Using CALSA+, we train a model with Reinforcement Learning with Verifiable Rewards (RLVR) to identify multiple logical interpretations and show that models trained with RLVR can perform on par with much bigger proprietary models. Our work is the first to attempt to annotate all the interpretations of logical structure on top of CAs. We publicly release our dataset to facilitate research in CA logical structure identification.
Citation
W. Wang et al., “Identification of Multiple Logical Interpretations in Counter-Arguments,” Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 6429–6444, 2025, doi: 10.18653/V1/2025.EMNLP-MAIN.326.
Source
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Conference
2025 Conference on Empirical Methods in Natural Language Processing
Keywords
Counter-Arguments Logical Structure, Multiple Logical Interpretations, Argument Mining, Dataset Creation: CALSA+, Reinforcement Learning with Verifiable Rewards, Expert Annotation in CAs, Logical Predicate Questions, Critical-Thinking Skill Enhancement
Subjects
Source
2025 Conference on Empirical Methods in Natural Language Processing
Publisher
Association for Computational Linguistics