Synergy between sufficient changes and sparse mixing procedure for disentangled representation learning
Li Zijian Fan Shunxing Zheng Yujia Ng Ignavier Xie Shaoan Chen Guangyi Dong Xinshuai Cai Ruichu Zhang Kun
Li Zijian
Fan Shunxing
Zheng Yujia
Ng Ignavier
Xie Shaoan
Chen Guangyi
Dong Xinshuai
Cai Ruichu
Zhang Kun
Supervisor
Department
Machine Learning
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Disentangled representation learning aims to uncover latent variables underlying the observed data, and generally speaking, rather strong assumptions are needed to ensure identifiability. Some approaches rely on sufficient changes on the distribution of latent variables indicated by auxiliary variables such as domain indices, but acquiring enough domains is often challenging. Alternative approaches exploit structural sparsity assumptions on the mixing procedure, but such constraints are usually (partially) violated in practice. Interestingly, we find that these two seemingly unrelated assumptions can actually complement each other to achieve identifiability. Specifically, when conditioned on auxiliary variables, the sparse mixing procedure assumption provides structural constraints on the mapping from estimated to true latent variables and hence compensates for potentially insufficient distribution changes. Building on this insight, we propose an identifiability theory with less restrictive constraints regarding distribution changes and the sparse mixing procedure, enhancing applicability to real-world scenarios. Additionally, we develop an estimation framework incorporating a domain encoding network and a sparse mixing constraint and provide two implementations, based on variational autoencoders and generative adversarial networks, respectively. Experiment results on synthetic and real-world datasets support our theoretical results. The code is available at https://github.com/jozerozero/Synergy Disentanglement. © 2025 13th International Conference on Learning Representations, ICLR 2025. All rights reserved.
Citation
Z. Li et al., “Synergy Between Sufficient Changes and Sparse Mixing Procedure for Disentangled Representation Learning,” International Conference on Representation Learning, vol. 2025, pp. 44065–44089, May 2025, Accessed: Jul. 22, 2025
Source
13th International Conference on Learning Representations, ICLR 2025
Conference
13th International Conference on Learning Representations, ICLR 2025
Keywords
Subjects
Source
13th International Conference on Learning Representations, ICLR 2025
Publisher
International Conference on Learning Representations, ICLR