Culture Through Figurative Language: Gaps and Bidirectional Links in LLM Cultural and Figurative Language Competence
Attia, Mena Mohammed
Attia, Mena Mohammed
Citations
Altmetric:
Author
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Thesis
Date
2026
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Figurative language and cultural knowledge are deeply intertwined: proverbs and idioms encode shared values and social norms that are intelligible only to those sufficiently immersed in the culture that produced them. Yet despite growing interest in Arabic NLP, the ability of large language models to understand and use Arabic figurative language, and its relationship to broader cultural competence, remains poorly understood. This thesis investigates the relationship between cultural knowledge and figurative language understanding in large language models (LLMs), pursuing two complementary questions: how well do LLMs understand and pragmatically use culturally grounded figurative expressions, and can fine-tuning on cultural data improve figurative understanding, and vice versa? Using figurative language as a proxy for cultural reasoning, evaluation tasks are designed that span contextual understanding, pragmatic use, and connotative interpretation in Arabic and English. Twenty-two open- and closed-source LLMs are evaluated on Egyptian Arabic idioms, multi-dialectal Arabic proverbs, and English proverbs, with the release of Kinayat, the first dataset of Egyptian Arabic idioms designed for figurative understanding and pragmatic use evaluation. Results reveal a consistent performance hierarchy: accuracy for Arabic proverbs falls 4.29% below English proverbs, and performance on Egyptian idioms trails Arabic proverbs by a further 10.28%. Pragmatic use proves harder still, with accuracy declining 14.07% relative to understanding tasks, though providing contextual idiomatic sentences recovers 10.66% of that gap. Models also struggle with connotative meaning, reaching at most 85.58% agreement with human annotators on idioms with 100% inter-annotator agreement. To explore whether the culture-figurative language connection can be leveraged to improve both competences, a systematic fine-tuning study is conducted across four models (ALLaM-7B, Fanar-1.9B, Qwen3-8B, and Llama-3.1-8B) and six Arabic datasets spanning cultural commonsense, proverbs, and poetry across diverse dialects and regions. Cross-domain transfer proves limited, inconsistent, and highly model-dependent: when cultural datasets serve as the training signal, average gains on idiom comprehension range from 2.44% to 3.78%, while figurative fine-tuning yields smaller gains on cultural benchmarks, ranging from 0.37% to 0.93%, though these two directions are not directly comparable given differences in datasets and task formulations. One notable finding emerges: fine-tuning on poetry produces consistent positive transfer to both idiom and proverb comprehension, suggesting that it develops a broader figurative competence---a sensitivity to non-literal meaning that transfers across figurative types rather than remaining tied to any one. A striking divide emerges between model families: Arabic-specialized models frequently regress after fine-tuning, suggesting prior saturation of relevant knowledge, whereas multilingual models show greater adaptation headroom. Error analysis further reveals that fine-tuning reinforces experiential cultural knowledge while destabilizing historically grounded factual knowledge. Taken together, these findings demonstrate that figurative language is an effective diagnostic for cultural reasoning, but that the relationship between culture and figurative competence, though conceptually natural, is not straightforwardly captured through fine-tuning alone, underscoring the need for more targeted approaches to culturally grounded language modeling.
Citation
Attia, Mena Mohammed, "Culture Through Figurative Language: Gaps and Bidirectional Links in LLM Cultural and Figurative Language Competence," M.S. Thesis, Natural Language Processing, MBZUAI, 2026.
Source
Conference
Keywords
LLMs, figurative language, culture