Linear Steerability in Language Models: When It Emerges and How It Evolves
She, Jianshu Li, Xinyue Xing, Eric Liu, Zhengzhong Ho, Qirong
She, Jianshu
Li, Xinyue
Xing, Eric
Liu, Zhengzhong
Ho, Qirong
Supervisor
Department
Machine Learning
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Language models can be steered by modifying their internal representations to control concepts such as emotion, style, or truthfulness in generation. However, the conditions for an effective intervention remain unclear and are often validated through heuristics and trial-and-error. To fill this gap, we demonstrate that intervention efficacy, measured by linear steerability (i.e., the ability to adjust output via linear transformations of hidden states), emerges during intermediate stages of training. Moreover, even closely related concepts (e.g., anger and sadness) exhibit steerability emergence at distinct stages of training*.To better interpret the dynamics of steerability during training, we adapt existing intervention techniques into a unified framework, referred to as the “Intervention Detector” (ID), which is designed to reveal how linear steerability evolves over the course of training through hidden state and representation analysis. ID reveals that concepts become increasingly linearly separable in the hidden space as training progresses, which strongly correlates with the emergence of linear steerability. We further introduce ID-based metrics, such as heatmaps, entropy trends, and cosine similarity, to help interpret how linear steerability evolves throughout training. In addition, we apply ID across different model families to ensure the generality of our findings on steerability dynamics.
Citation
J. She, X. Li, E. P. Xing, Z. Liu, and Q. Ho, “Linear Steerability in Language Models: When It Emerges and How It Evolves,” Findings of the Association for Computational Linguistics: EMNLP 2025, pp. 17821–17846, 2025, doi: 10.18653/V1/2025.FINDINGS-EMNLP.969
Source
Findings of the Association for Computational Linguistics
Conference
2025 Conference on Empirical Methods in Natural Language Processing
Keywords
Subjects
Source
2025 Conference on Empirical Methods in Natural Language Processing
Publisher
Association for Computational Linguistics