Continual Learning and Unknown Object Discovery in 3D Scenes via Self-distillation
Boudjoghra, Mohamed El Amine Lahoud, Jean Cholakkal, Hisham Anwer, Rao Muhammad Khan, Salman Khan, Fahad Shahbaz
Boudjoghra, Mohamed El Amine
Lahoud, Jean
Cholakkal, Hisham
Anwer, Rao Muhammad
Khan, Salman
Khan, Fahad Shahbaz
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Open-world 3D instance segmentation is a recently introduced problem with diverse applications, notably in continually learning embodied agents. This task involves segmenting unknown instances and learning new instances when their labels are introduced. However, prior research in the open-world domain has traditionally addressed the two sub-problems, namely continual learning and unknown object identification, separately. This approach has resulted in limited performance on unknown instances and cannot effectively mitigate catastrophic forgetting. Additionally, these methods bypass the utilization of the information stored in the previous version of the continual learning model, instead relying on a dedicated memory to store historical data samples, which inevitably leads to an expansion of the memory budget. In this paper, we argue that continual learning and unknown object identification are desired to be tackled in conjunction. To this end, we propose a new exemplar-free approach for 3D continual learning and unknown object discovery through continual self-distillation. Our approach, named OpenDistill3D, leverages the pseudo-labels generated by the model from the preceding task to improve the unknown predictions during training while simultaneously mitigating catastrophic forgetting. By integrating these pseudo-labels into the continual learning process, we achieve enhanced performance in handling unknown objects. We validate the efficacy of the proposed approach via comprehensive experiments on various splits of the ScanNet200 dataset, showcasing superior performance in contin vual learning and unknown object retrieval compared to the state-of-the-art. Code and model are available at github.com/aminebdj/OpenDistill3D.
Citation
M. E. A. Boudjoghra, J. Lahoud, H. Cholakkal, R. M. Anwer, S. Khan, and F. S. Khan, “Continual Learning and Unknown Object Discovery in 3D Scenes via Self-distillation,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , vol. 15131 LNCS, pp. 416–431, 2025, doi: 10.1007/978-3-031-73464-9_25.
Source
Computer Vision – ECCV 2024
Conference
Keywords
Adversarial machine learning, Budget control, Federated learning, Self-supervised learning, 3D scenes, Catastrophic forgetting, Continual learning, Diverse applications
Subjects
Source
Publisher
Springer Nature