Towards generalizable reinforcement learning via causality-guided self-adaptive representations
Yang, Yupei Huang, Biwei Feng, Fan Wang, Xinyue Tu, Shikui Xu, Lei
Yang, Yupei
Huang, Biwei
Feng, Fan
Wang, Xinyue
Tu, Shikui
Xu, Lei
Supervisor
Department
Machine Learning
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
General intelligence requires quick adaptation across tasks. While existing reinforcement learning (RL) methods have made progress in generalization, they typically assume only distribution changes between source and target domains. In this paper, we explore a wider range of scenarios where not only the distribution but also the environment spaces may change. For example, in the CoinRun environment, we train agents from easy levels and generalize them to difficulty levels where there could be new enemies that have never occurred before. To address this challenging setting, we introduce a causality-guided self-adaptive representation-based approach, called CSR, that equips the agent to generalize effectively across tasks with evolving dynamics. Specifically, we employ causal representation learning to characterize the latent causal variables within the RL system. Such compact causal representations uncover the structural relationships among variables, enabling the agent to autonomously determine whether changes in the environment stem from distribution shifts or variations in space, and to precisely locate these changes. We then devise a three-step strategy to fine-tune the causal model under different scenarios accordingly. Empirical experiments show that CSR efficiently adapts to the target domains with only a few samples and outperforms state-of-the-art baselines on a wide range of scenarios, including our simulated environments, CartPole, CoinRun and Atari games. © 2025 13th International Conference on Learning Representations, ICLR 2025. All rights reserved.
Citation
Y. Yang, B. Huang, F. Feng, X. Wang, S. Tu, and L. Xu, “Towards Generalizable Reinforcement Learning via Causality-Guided Self-Adaptive Representations,” International Conference on Representation Learning, vol. 2025, pp. 53052–53083, May 2025
Source
13th International Conference on Learning Representations, ICLR 2025
Conference
13th International Conference on Learning Representations, ICLR 2025
Keywords
Subjects
Source
13th International Conference on Learning Representations, ICLR 2025
Publisher
International Conference on Learning Representations, ICLR