Item

Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs

Alwajih, Fakhraddin
; 
El Mekki, Abdellah
; 
Magdy, Samar Mohamed
; 
Elmadany, Abdelrahim A.
; 
Nacar, Omer
; 
Nagoudi, ElMoatez Billah
; 
Abdel-Salam, Reem
; 
Atwany, Hanin
; 
Nafea, Youssef
; 
Yahya, Abdulfattah Mohammed
... show 10 more
Author
Alwajih, Fakhraddin
El Mekki, Abdellah
Magdy, Samar Mohamed
Elmadany, Abdelrahim A.
Nacar, Omer
Nagoudi, ElMoatez Billah
Abdel-Salam, Reem
Atwany, Hanin
Nafea, Youssef
Yahya, Abdulfattah Mohammed
Alhamouri, Yahya Rahaf
Alsayadi, Hamzah A.
Zayed, Hiba
Shatnawi, Sara
Sibaee, Serry
Ech-Chammakhy, Yasir
Al-Dhabyani, Walid
Ali, Marwa Mohamed
Jarraya, Imen
El-Shangiti, Ahmed Oumar
Alraeesi, Aisha
Al-Ghrawi, Mohammed Anwar
Al-Batati, Abdulrahman S.
Mohamed, Elgizouli
Elgindi, Noha Taha
Saeed, Muhammed
Atou, Houdaifa
Yahia, Issam Ait
Bouayad, Abdelhak
Machrouh, Mohammed
Makouar, Amal
Alkawi, Dania
Mohamed, Mukhtar
Abdelfadil, Safaa Taher
Ounnoughene, Amine Ziad
Anfel, Rouabhia
Assi, Rwaa
Sorkatti, Ahmed
Tourad, Mohamedou Cheikh
Koubaa, Anis
Berrada, Ismail
Jarrar, Mustafa
Shehata, Shady
Abdul-Mageed, Muhammad
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Publications
Show all metadata
Research Projects
Organizational Units
Journal Issue
Abstract
As large language models (LLMs) become increasingly integrated into daily life, ensuring their cultural sensitivity and inclusivity is paramount. We introduce PALM, a year-long community-driven project covering all 22 Arab countries. The dataset includes instructions (input, response pairs) in both Modern Standard Arabic (MSA) and dialectal Arabic (DA), spanning 20 diverse topics. Built by a team of 44 researchers across the Arab world, all of whom are authors of this paper, PALM offers a broad, inclusive perspective. We use PALM to evaluate the cultural and dialectal capabilities of several frontier LLMs, revealing notable limitations. For instance, while closed-source LLMs generally exhibit strong performance, they are not without flaws, and smaller open-source models face greater challenges. Moreover, certain countries (e.g., Egypt, the UAE) appear better represented than others (e.g., Iraq, Mauritania, Yemen). Our annotation guidelines, code, and data for reproducibility are publicly available. More information about PALM is available at our project page: https://github.com/UBC-NLP/palm.
Citation
F. Alwajih et al., “Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs,” Proceedings of the Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 32871–32894, 2025, doi: 10.18653/V1/2025.ACL-LONG.1579.
Source
PROCEEDINGS OF THE 63RD ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 1: LONG PAPERS
Conference
63rd Association for Computational Linguistics Meeting-ACL-Annual
Keywords
Arabic Language Models, Cultural Inclusivity, Dataset Creation, Modern Standard Arabic, Dialectal Arabic, Language Model Evaluation, Cultural Diversity, Instruction Tuning
Subjects
Source
63rd Association for Computational Linguistics Meeting-ACL-Annual
Publisher
Association for Computational Linguistics
DOI
10.18653/v1/2025.acl-long.1579
Additional links
https://aclanthology.org/2025.acl-long.1579/
Full-text link