Nonparametric Factor Analysis and Beyond
Zheng, Yujia Liu, Yang Yao, Jiaxiong Hu, Yingyao Zhang, Kun
Zheng, Yujia
Liu, Yang
Yao, Jiaxiong
Hu, Yingyao
Zhang, Kun
Supervisor
Department
Machine Learning
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Nearly all identifiability results in unsupervised representation learning inspired by, e.g., independent component analysis, factor analysis, and causal representation learning, rely on assumptions of additive independent noise or noiseless regimes. In contrast, we study the more general case where noise can take arbitrary forms, depend on latent variables, and be non-invertibly entangled within a nonlinear function. We propose a general framework for identifying latent variables in the nonparametric noisy settings. We first show that, under suitable conditions, the generative model is identifiable up to certain submanifold indeterminacies even in the presence of non-negligible noise. Furthermore, under the structural or distributional variability conditions, we prove that latent variables of the general nonlinear models are identifiable up to trivial indeterminacies. Based on the proposed theoretical framework, we have also developed corresponding estimation methods and validated them in various synthetic and real-world settings. Interestingly, our estimate of the true GDP growth from alternative measurements suggests more insightful information on the economies than official reports. We expect our framework to provide new insight into how both researchers and practitioners deal with latent variables in real-world scenarios.
Citation
Y. Zheng, Y. Liu, J. Yao, Y. Hu, and K. Zhang, “Nonparametric Factor Analysis and Beyond,” in Proc. 28th Int. Conf. Artif. Intell. Stat. (AISTATS 2025), vol. 258, Proc. Mach. Learn. Res., Mai Khao, Thailand, May 3–5, 2025, pp. 424–432.
Source
Proceedings of Machine Learning Research
Conference
28th International Conference on Artificial Intelligence and Statistics, AISTATS 2025
Keywords
Economics, Factor Analysis, Multivariant Analysis, Nonlinear Analysis, Nonlinear Simulations, Unsupervised Learning, Factors Analysis, Generative Model, Identifiability, Independent Components Analysis, Independent Noise, Latent Variable, Nonlinear Functions, Nonparametrics, Submanifolds, Suitable Conditions, Independent Component Analysis
Subjects
Source
28th International Conference on Artificial Intelligence and Statistics, AISTATS 2025
Publisher
ML Research Press