Loading...
Cross Platform MultiModal Retrieval Augmented Distillation for Code-Switched Content Understanding
Thapa, Surendrabikram Veeramani, Hariram Razzak, Imran Lee, Roy Ka-Wei Naseem, Usman
Thapa, Surendrabikram
Veeramani, Hariram
Razzak, Imran
Lee, Roy Ka-Wei
Naseem, Usman
Supervisor
Department
Computational Biology
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
In the era of digital communication and social media, the proliferation of multimodal content, such as code-switched memes, has become a ubiquitous form of expression. This phenomenon is especially significant for low-resource languages like Nepali, where the need for sentiment analysis and hate speech detection remains unmet due to the unavailability of publicly available datasets. To address this gap, we provide ENeMeme, an annotated dataset of 4, 211 code-switched memes in the Nepali-English language for sentiment and hate speech. While the previous state-of-the-art methods of meme analysis particularly focus on high-resource language, they fail to perform well in low-resource language. To bridge this gap, our paper also builds on existing literature to adapt a novel multimodal model, MM-RAD, designed to understand code-switched Nepali-English memes, leveraging both textual and visual content. The model's effectiveness is analyzed across various retrieval platforms. Our proposed MM-RAD demonstrates superior performance in sentiment analysis and hate speech detection compared to individual baseline models. The dataset can be availed through https://github.com/therealthapa/crossplatform. © 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM.
Citation
S. Thapa, H. Veeramani, I. Razzak, R. K.-W. Lee, and U. Naseem, “Cross Platform MultiModal Retrieval Augmented Distillation for Code-Switched Content Understanding,” vol. 10, pp. 2042–2051, May 2025, doi: 10.1145/3701716.3718389
Source
WWW Companion 2025 - Companion Proceedings of the ACM Web Conference 2025
Conference
34th ACM Web Conference, WWW Companion 2025
Keywords
Code-Switched Languages, Meme Analysis, Multimodal Processing, Natural Language Processing
Subjects
Source
34th ACM Web Conference, WWW Companion 2025
Publisher
Association for Computing Machinery