Loading...
Aligned Probing: Relating Toxic Behavior and Model Internals
Waldis, Andreas Gautam, Vagrant Lauscher, Anne Klakow, Dietrich Gurevych, Iryna
Waldis, Andreas
Gautam, Vagrant
Lauscher, Anne
Klakow, Dietrich
Gurevych, Iryna
Files
Loading...
tacl.a.613.pdf
Adobe PDF, 1.73 MB
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Journal article
Date
License
http://creativecommons.org/licenses/by/4.0/
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Abstract Warning: This paper contains offensive text. We introduce aligned probing, a novel interpretability framework that aligns the behavior of language models (LMs), based on their outputs, and their internal representations (internals). Using this framework, we examine over 20 OLMo, Llama, and Mistral models, bridging behavioral and internal perspectives for toxicity for the first time. Our results show that LMs strongly encode information about the toxicity level of inputs and subsequent outputs, particularly in lower layers. Focusing on how unique LMs differ offers both correlative and causal evidence that they generate less toxic output when strongly encoding information about the input toxicity. We also highlight the heterogeneity of toxicity, as model behavior and internals vary across unique attributes such as Threat. Finally, four case studies analyzing detoxification, multi-prompt evaluations, model quantization, and pre-training dynamics underline the practical impact of aligned probing with further concrete insights. Our findings contribute to a more holistic understanding of LMs, both within and beyond the context of toxicity. alignedprobing.github.io
Citation
A. Waldis, V. Gautam, A. Lauscher, D. Klakow, I. Gurevych, "Aligned Probing: Relating Toxic Behavior and Model Internals," Transactions of the Association for Computational Linguistics, vol. 14, pp. 271-291, 2026, https://doi.org/10.1162/tacl.a.613.
Source
Transactions of the Association for Computational Linguistics
Conference
Keywords
46 Information and Computing Sciences, 4602 Artificial Intelligence
Subjects
Source
Publisher
MIT Press