Grounding Fallacies Misrepresenting Scientific Publications in Evidence
Glockner, Max Hou, Yufang Nakov, Preslav Gurevych, Iryna
Glockner, Max
Hou, Yufang
Nakov, Preslav
Gurevych, Iryna
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Health-related misinformation claims often falsely cite a credible biomedical publication as evidence. These publications only superficially seem to support the false claim, when logical fallacies are applied. In this work, we aim to detect and to highlight such fallacies, which requires assessing the exact content of the misrepresented publications. To achieve this, we introduce MISSCIPLUS, an extension of the fallacy detection dataset MISSCI. MISSCIPLUS extends MISSCI by grounding the applied fallacies in real-world passages from misrepresented studies. This creates a realistic test-bed for detecting and verbalizing fallacies under real-world input conditions, and enables new and realistic passage-retrieval tasks. MISSCIPLUS is the first logical fallacy dataset which pairs the real-world misrepresented evidence with incorrect claims, identical to the input to evidence-based fact-checking models. With MISSCIPLUS, we i) benchmark retrieval models in identifying passages that support claims only with fallacious reasoning, ii) evaluate how well LLMs verbalize fallacious reasoning based on misrepresented scientific passages, and iii) assess the effectiveness of fact-checking models in refuting claims that misrepresent biomedical research. Our findings show that current fact-checking models struggle to use misrepresented scientific passages to refute misinformation. Moreover, these passages can mislead LLMs into accepting false claims as true.(1)
Citation
M. Glockner, Y. Hou, P. Nakov, and I. Gurevych, “Grounding Fallacies Misrepresenting Scientific Publications in Evidence,” vol. 1, pp. 9732–9767, Jun. 2025, doi: 10.18653/V1/2025.NAACL-LONG.491.
Source
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Conference
2025 Conference of the North American Chapter of the Association for Computational Linguistics-NAACL
Keywords
Scientific Misinformation, Fallacy Detection, Fact-Checking, Biomedical Evidence, Passage Retrieval, Logical Fallacies, Large Language Models, Misrepresented Scientific Claims
Subjects
Source
2025 Conference of the North American Chapter of the Association for Computational Linguistics-NAACL
Publisher
Association for Computational Linguistics