Loading...
KG-UQ: Knowledge Graph-Based Uncertainty Quantification for Long Text in Large Language Models
Yuan, Yingqing Tao, Linwei Lu, Haohui Khushi, Matloob Razzak, Imran Dras, Mark Yang, Jian Naseem, Usman
Yuan, Yingqing
Tao, Linwei
Lu, Haohui
Khushi, Matloob
Razzak, Imran
Dras, Mark
Yang, Jian
Naseem, Usman
Supervisor
Department
Computational Biology
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
With the commercialization of large language models (LLMs) and their integration into daily life, addressing their susceptibility to hallucinations-unfactual information in generated outputs-has become an urgent priority. Existing uncertainty quantification (UQ) methods often rely on access to LLMs' internal states, which is unavailable for closed-source models like GPTs, or are primarily designed for short text. Current research on long text typically evaluates sentences individually, overlooking smaller semantic units that better capture the text's complexity. Recognizing the potential of knowledge graphs (KGs) to extract structured relationships from unstructured text, we propose KG-UQ, a UQ method leveraging KGs to address the semantic intricacies of long text. Our approach involves constructing KGs from long-text outputs and utilizing their embeddings to estimate uncertainties. Through our analysis, we demonstrate that knowledge graphs are an effective tool for decomposing long text into fundamental statements. However, we also highlight the increased uncertainty introduced during KG construction, stemming from inherent challenges in accurately capturing all semantic information. © 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM.
Citation
Y. Yuan et al., “KG-UQ: Knowledge Graph-Based Uncertainty Quantification for Long Text in Large Language Models,” pp. 2071–2077, May 2025, doi: 10.1145/3701716.3717660
Source
WWW Companion 2025 - Companion Proceedings of the ACM Web Conference 2025
Conference
34th ACM Web Conference, WWW Companion 2025
Keywords
Hallucinations, Knowledge Graphs, Large Language Models, Uncertainty Quantification
Subjects
Source
34th ACM Web Conference, WWW Companion 2025
Publisher
Association for Computing Machinery