Beyond Fluency: Aligning Arabic LLMs Through Domain Knowledge and Cultural Steering
Elbadry, Rania Hossam Elmohamady
Elbadry, Rania Hossam Elmohamady
Citations
Altmetric:
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Thesis
Date
2026
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
The rapid adoption of Large Language Models (LLMs) in Arabic-language contexts demands both cultural sensitivity and domain expertise, yet current models fall short on both fronts. Mod- els default to Western cultural norms when explicit country cues are absent, and models that appear fluent in Arabic lack the specialized reasoning required for high-stakes domains such as Islamic finance and Shari’ah-compliant decision-making. While Arabic-centric LLMs have been developed, they are evaluated only on generic benchmarks that ignore these failures.
To address the cultural alignment gap, we propose YaPO (Yet another Policy Optimization), a reference-free method that learns sparse steering vectors in the latent space of a sparse au- toencoder (SAE) using a bi-directional preference optimization objective. Unlike dense steering approaches that entangle multiple latent factors due to neuron multi-semanticity, YaPO produces disentangled, interpretable, and efficient steering directions. On a newly curated cultural bench- mark spanning five language families and fifteen cultural contexts, YaPO achieves the highest ro- bust cultural accuracy across languages, converges an order of magnitude faster than dense base- lines, and generalizes to hallucination mitigation, jailbreak defense, and power-seeking behaviors with no degradation on the Massive Multitask Language Understanding (MMLU) benchmark.
To address the domain knowledge gap, we introduce SAHM, the first Arabic financial Nat- ural Language Processing (NLP) benchmark spanning seven tasks: question answering (QA) over standards issued by the Accounting and Auditing Organization for Islamic Financial Insti- tutions (AAOIFI), fatwa-based QA and multiple-choice questions (MCQ), accounting and busi- ness exams, financial sentiment analysis, extractive summarization, and event–cause reasoning, comprising 14,380 expert-verified instances from authentic regulatory, juristic, and corporate sources. Evaluating 20 LLMs, we find that Arabic fluency does not imply financial reasoning: models achieving 91% accuracy on recognition tasks degrade sharply on generation, and event– cause reasoning exposes the widest performance gap, with scores ranging from 1.89 to 9.84 on a 10-point scale. A targeted 7B instruction-tuned model surpasses GPT-5 by over 20 points on business, demonstrating that domain-specific instruction-tuning outperforms scale alone.
Together, these contributions show that robust Arabic LLM alignment requires progress along two orthogonal axes behavioral steering at inference time and knowledge injection through tar- geted training neither of which subsumes the other.
Citation
Elbadry, Rania Hossam Elmohamady, "Beyond Fluency: Aligning Arabic LLMs Through Domain Knowledge and Cultural Steering," M.S. Thesis, Natural Language Processing, MBZUAI, 2026.
Source
Conference
Keywords
Cultural Alignment, Arabic NLP, Large Language Models, Islamic Finance, Activation Steering, Sparse Autoencoders