Loading...
Identifying Latent State-Transition Processes for Individualized Reinforcement Learning
Sun, Yuewen Huang, Biwei Yao, Yu Zeng, Donghuo Dong, Xinshuai Jin, Songyao Sun, Boyang Legaspi, Roberto Ikeda, Kazushi Spirtes, Peter
Sun, Yuewen
Huang, Biwei
Yao, Yu
Zeng, Donghuo
Dong, Xinshuai
Jin, Songyao
Sun, Boyang
Legaspi, Roberto
Ikeda, Kazushi
Spirtes, Peter
Supervisor
Department
Machine Learning
Embargo End Date
Type
Conference proceeding
Date
2024
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
The application of reinforcement learning (RL) involving interactions with individuals has grown significantly in recent years. These interactions, influenced by factors such as personal preferences and physiological differences, causally influence state transitions, ranging from health conditions in healthcare to learning progress in education. As a result, different individuals may exhibit different state-transition processes. Understanding individualized state-transition processes is essential for optimizing individualized policies. In practice, however, identifying these state-transition processes is challenging, as individual-specific factors often remain latent. In this paper, we establish the identifiability of these latent factors and introduce a practical method that effectively learns these processes from observed state-action trajectories. Experiments on various datasets show that the proposed method can effectively identify latent state-transition processes and facilitate the learning of individualized RL policies.
Citation
Y. Sun et al., “Identifying Latent State-Transition Processes for Individualized Reinforcement Learning,” Adv Neural Inf Process Syst, vol. 37, pp. 124887–124924, Dec. 2024.
Source
Advances in Neural Information Processing Systems (NeurIPS 2024)
Conference
Keywords
Individualized reinforcement learning, Latent state-transition processes, Identifiability, State-action trajectories, Personalized policy optimization
Subjects
Source
Publisher
NEURIPS