Enhancing Small-Scale Language Models to Approach LLM Performance for RAG-based QA
Lialin, Ilia
Lialin, Ilia
Author
Supervisor
Department
Natural Language Processing
Embargo End Date
30/05/2025
Type
Thesis
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
The rapid rise of large language models (LLMs) has advanced NLP but comes with high computational costs. Retrieval Augmented Generation (RAG) offers efficiency by integrating external knowledge, yet most systems still depend on large models. This thesis explores whether smaller models can achieve competitive performance in RAG settings when supported by carefully designed pipelines. Experiments span both general and domain-specific datasets, including regulatory texts, to evaluate performance and generalisation. Additionally, the study highlights the limitations of traditional evaluation metrics like BLEU and ROUGE, which often fail to reflect semantic quality. It advocates for reference-free evaluation as a better fit for modern generative models. Results demonstrate that smaller models, when properly structured and evaluated, can provide scalable and costeffective alternatives for real-world NLP applications.
Citation
Ilia Lialin, “Enhancing Small-Scale Language Models to Approach LLM Performance for RAG-based QA,” Master of Science thesis, Natural Language Processing, MBZUAI, 2025.
Source
Conference
Keywords
Retrieval-Augmented Generation (RAG), Large Language Models (LLM), Reference-Free Evaluation, Natural Language Processing (NLP), Information Retrieval, Question Answering