Evaluating Evidence Attribution in Generated Fact Checking Explanations
Xing, Rui Baldwin, Timothy Lau, Jey Han
Xing, Rui
Baldwin, Timothy
Lau, Jey Han
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Automated fact-checking systems often struggle with trustworthiness, as their generated explanations can include hallucinations. In this work, we explore evidence attribution for fact-checking explanation generation. We introduce a novel evaluation protocol - citation masking and recovery - to assess attribution quality in generated explanations. We implement our protocol using both human annotators and automatic annotators, and find that LLM annotation correlates with human annotation, suggesting that attribution assessment can be automated. Finally, our experiments reveal that: (1) the best-performing LLMs still generate explanations with inaccurate attributions; and (2) human-curated evidence is essential for generating better explanations. Code and data are available here: https://github.com/ruixing76/Transparent-FCExp.
Citation
R. Xing, T. Baldwin, and J. H. Lau, “Evaluating Evidence Attribution in Generated Fact Checking Explanations,” vol. 1, pp. 5475–5496, Jun. 2025, doi: 10.18653/V1/2025.NAACL-LONG.282
Source
Proceedings of the 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies: Long Papers, NAACL-HLT 2025
Conference
2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2025
Keywords
Subjects
Source
2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2025
Publisher
Association for Computational Linguistics