BERTastic at SemEval-2025 Task 10: State-of-the-Art Accuracy in Coarse-Grained Entity Framing for Hindi News
Mahmoud, Tarek Xie, Zhuohan Nakov, Preslav
Mahmoud, Tarek
Xie, Zhuohan
Nakov, Preslav
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
We describe our system for SemEval-2025 Task 10 Subtask 1 on coarse-grained entity framing in Hindi news, exploring two complementary strategies. First, we experiment with LLM prompting using GPT-4o, comparing hierarchical multi-step prompting with native single-step prompting for both main and fine-grained role prediction. Second, we conduct an extensive study on fine-tuning XLM-R, analyzing different context granularities (full article, paragraph, or sentence-level entity mentions), monolingual vs. multilingual settings, and main vs. fine-grained role labels. Our best system, trained on fine-grained role annotations across languages using sentence-level context, achieved 43.99% exact match, 56.56 % precision, 47.38% recall, and 51.57% F1-score. Notably, our system set a new state-of-the-art for main role prediction on Hindi news, achieving 78.48 % accuracy - outperforming the next best model at 76.90%, as per the official leaderboard. Our findings highlight effective strategies for entity framing in multilingual and low-resource settings.
Citation
T. Mahmoud, Z. Xie, and P. Nakov, “BERTastic at SemEval-2025 Task 10: State-of-the-Art Accuracy in Coarse-Grained Entity Framing for Hindi News,” 2025. [Online]. Available: https://aclanthology.org/2025.semeval-1.55/
Source
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
Conference
19th International Workshop on Semantic Evaluation (SemEval-2025)
Keywords
Subjects
Source
19th International Workshop on Semantic Evaluation (SemEval-2025)
Publisher
Association for Computational Linguistics