Analysis of Internal Representations of Knowledge with Expressions of Familiarity
Tanaka, Kenshiro Sakai, Yoshihiro Zhao, Yufeng Inoue, Naoya Sato, Kai Takahashi, Ryosuke Heinzerling, Benjamin Inui, Kentaro
Tanaka, Kenshiro
Sakai, Yoshihiro
Zhao, Yufeng
Inoue, Naoya
Sato, Kai
Takahashi, Ryosuke
Heinzerling, Benjamin
Inui, Kentaro
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
Japanese
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Research on the ability of large language models (LLMs) to judge the familiarity of knowledge is progressing. However, little attention has been given to whether LLMs can assess the familiarity of knowledge when linguistic expressions such as "It is known that..." are included during training. This study investigates how familiarity is internally represented in LLMs. To achieve this, we trained the models on descriptions of knowledge accompanied by linguistic expressions indicating familiarity. The internal representations of the models were then analyzed. The findings reveal that (1) familiarity information is separately retained in the internal representations of knowledge, for the linguistic expressions provided during training, and (2) familiarity information is separately maintained for each position of the linguistic expressions. This study provides a foundation for understanding the mechanisms underlying LLMs' ability to judge familiarity.
Citation
K. Tanaka et al., “Analysis of Internal Representations of Knowledge with Expressions of Familiarity,” pp. 2Win523-2Win523, 2025, doi: 10.11517/PJSAI.JSAI2025.0_2WIN523
Source
Proceedings of the Annual Conference of JSAI, 2025
Conference
The 39th Annual Conference of the Japanese Society for Artificial Intelligence
Keywords
language models, knowledge representation, familiarity judgement
Subjects
Source
The 39th Annual Conference of the Japanese Society for Artificial Intelligence
Publisher
Japanese Society for Artificial Intelligence