Loading...
Unveiling the Influence of Amplifying Language-Specific Neurons
Rahmanisa, Inaya Andrylie, Lyzander Marciano Ihsani, Mahardika Krisna Wicaksono, Alfan Farizki Wibowo, Haryo Akbarianto Aji, Alham Fikri
Rahmanisa, Inaya
Andrylie, Lyzander Marciano
Ihsani, Mahardika Krisna
Wicaksono, Alfan Farizki
Wibowo, Haryo Akbarianto
Aji, Alham Fikri
Files
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Conference proceeding
Date
License
http://creativecommons.org/licenses/by/4.0/
Language
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Language-specific neurons, units in LLMs that strongly correlate with individual languages have been shown to influence model behavior by deactivating them. However, their role in amplification remains underexplored.This work investigates the effect of amplifying language-specific neurons through interventions across 18 languages, including low-resource ones, using three models primarily trained in different languages. We compare amplification factors by their effectiveness in steering to the target language using a proposed Language Steering Shift (LSS) evaluation score, then evaluate it on downstream tasks: commonsense reasoning (XCOPA, XWinograd), knowledge (Include), and translation (FLORES). The optimal amplification steering factors effectively steer output toward nearly all tested languages. Intervention using this factor on downstream tasks improves self-language performance in some cases but generally degrades cross-language results. These findings highlight the effect of language-specific neurons in multilingual behavior, where amplification can be beneficial especially for low-resource languages, but provides limited advantage for cross-lingual transfer.
Citation
I. Rahmanisa, L.M. Andrylie, M.K. Ihsani, A.F. Wicaksono, H.A. Wibowo, A.F. Aji, "Unveiling the Influence of Amplifying Language-Specific Neurons," 2025, pp. 919-968.
Source
Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics
Conference
Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics
Keywords
Subjects
Source
Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics
Publisher
Association for Computational Linguistics