Towards Efficient General Feature Prediction in Masked Skeleton Modeling
Sun, Shengkai Zhang, Zefan Dong, Jianfeng Cheng, Zhiyong Chang, Xiaojun Wang, Meng
Sun, Shengkai
Zhang, Zefan
Dong, Jianfeng
Cheng, Zhiyong
Chang, Xiaojun
Wang, Meng
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
License
Language
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Recent advances in the masked autoencoder (MAE) paradigm have significantly propelled self-supervised skeleton-based action recognition. However, most existing approaches limit reconstruction targets to raw joint coordinates or their simple variants, resulting in computational redundancy and limited semantic representation. To address this, we propose a novel General Feature Prediction framework (GFP) for efficient mask skeleton modeling. Our key innovation is replacing conventional low-level reconstruction with high-level feature prediction that spans from local motion patterns to global semantic representations. Specifically, we introduce a collaborative learning framework where a lightweight target generation network dynamically produces diversified supervision signals across spatial-temporal hierarchies, avoiding reliance on pre-computed offline features. The framework incorporates constrained optimization to ensure feature diversity while preventing model collapse. Experiments on $N T U \text{RGB}+D$ 60, NTU RGB+D 120 and PKU-MMD demonstrate the benefits of our approach: Computational efficiency (with $6.2 \times$ faster training than standard masked skeleton modeling methods) and superior representation quality, achieving state-of-the-art performance in various downstream tasks.
Citation
S. Sun, Z. Zhang, J. Dong, Z. Cheng, X. Chang, M. Wang, "Towards Efficient General Feature Prediction in Masked Skeleton Modeling," 2026, pp. 12212-12221.
Source
2025 IEEE/CVF International Conference on Computer Vision (ICCV)
Conference
2025 IEEE/CVF International Conference on Computer Vision (ICCV)
Keywords
46 Information and Computing Sciences, 4603 Computer Vision and Multimedia Computation, 4607 Graphics, Augmented Reality and Games, 4611 Machine Learning
Subjects
Source
2025 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher
IEEE