Mechanistic Unveiling of Transformer Circuits: Self-Influence as a Key to Model Reasoning
Zhang, Lin Hu, Lijie Wang, Di
Zhang, Lin
Hu, Lijie
Wang, Di
Author
Supervisor
Department
Machine Learning
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Transformer-based language models have achieved significant success; however, their internal mechanisms remain largely opaque due to the complexity of non-linear interactions and high-dimensional operations. While previous studies have demonstrated that these models implicitly embed reasoning trees, humans typically employ various distinct logical reasoning mechanisms to complete the same task. It is still unclear which multi-step reasoning mechanisms are used by language models to solve such tasks. In this paper, we aim to address this question by investigating the mechanistic interpretability of language models, particularly in the context of multi-step reasoning tasks. Specifically, we employ circuit analysis and self-influence functions to evaluate the changing importance of each token throughout the reasoning process, allowing us to map the reasoning paths adopted by the model. We apply this methodology to the GPT-2 model on a prediction task (IOI) and demonstrate that the underlying circuits reveal a human-interpretable reasoning process used by the model.
Citation
L. Zhang, L. Hu, and D. Wang, “Mechanistic Unveiling of Transformer Circuits: Self-Influence as a Key to Model Reasoning,” pp. 1387–1404, Jun. 2025, doi: 10.18653/V1/2025.FINDINGS-NAACL.76
Source
Findings of the Association for Computational Linguistics: NAACL 2025
Conference
Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics
Keywords
Subjects
Source
Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics
Publisher
Association for Computational Linguistics