Uncertainty Quantification for Large Language Models
Panov, Maxim Shelmanov, Artem Vashurin, Roman Vazhentsev, Artem Fadeeva, Ekaterina Rvanova, Lyudmila Baldwin, Timothy
Panov, Maxim
Shelmanov, Artem
Vashurin, Roman
Vazhentsev, Artem
Fadeeva, Ekaterina
Rvanova, Lyudmila
Baldwin, Timothy
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Conference proceeding
Date
License
Language
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Large language models (LLMs) power many NLP applications; yet, they can produce fluent but incorrect content (hallucinations), which threatens reliability and user trust. This tutorial introduces uncertainty quantification (UQ) for text generation: methods that attach an explicit reliability signal to model outputs and enable practical safeguards such as hallucination detection and selective generation. We begin with core uncertainty concepts and explain why techniques that work well for classification do not directly transfer to autoregressive generation. We then survey representative white-box and black-box approaches, from entropy- and probability-based scores to learned probes that leverage internal representations.Retrieval-augmented generation (RAG) has become a core design pattern for LLM applications. Incorporating retrieved evidence introduces both new challenges and valuable structures for uncertainty estimation. In the ECIR edition of the tutorial, we focus on UQ techniques tailored to RAG pipelines and briefly discuss how uncertainty can guide agentic workflows.Practical demonstrations are done using LM-Polygraph (https://github.com/IINemo/lm-polygraph), an open-source toolkit that consolidates more than forty recent UQ and calibration methods and provides a large-scale benchmark, making it easy to reproduce results and integrate UQ into applications with minimal code. Overall, the tutorial is intended to lower the barrier to entry for researchers and developers who want to evaluate existing UQ methods, design improved ones, and deploy uncertainty-aware LLM systems.
Citation
M. Panov, A. Shelmanov, R. Vashurin, A. Vazhentsev, E. Fadeeva, L. Rvanova , et al., "Uncertainty Quantification for Large Language Models," 2026, pp. 53-59.
Source
Lecture Notes in Computer Science
Conference
48th European Conference on Information Retrieval, ECIR 2026
Keywords
46 Information and Computing Sciences, 4605 Data Management and Data Science
Subjects
Source
48th European Conference on Information Retrieval, ECIR 2026
Publisher
Springer Nature