GaussCtrl: Multi-view Consistent Text-Driven 3D Gaussian Splatting Editing
Wu, Jing Bian, Jia-Wang Li, Xinghui Wang, Guangrun Reid, Ian Torr, Philip Prisacariu, Victor Adrian
Wu, Jing
Bian, Jia-Wang
Li, Xinghui
Wang, Guangrun
Reid, Ian
Torr, Philip
Prisacariu, Victor Adrian
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
We propose GaussCtrl, a text-driven method to edit a 3D scene reconstructed by the 3D Gaussian Splatting (3DGS). Our method first renders a collection of images by using the 3DGS and edits them by using a pre-trained 2D diffusion model (ControlNet) based on the input prompt, which is then used to optimise the 3D model. Our key contribution is multi-view consistent editing, which enables editing all images together instead of iteratively editing one image while updating the 3D model as in previous works. It leads to faster editing as well as higher visual quality. This is achieved by the two terms: (a) depth-conditioned editing that enforces geometric consistency across multi-view images by leveraging naturally consistent depth maps. (b) attention-based latent code alignment that unifies the appearance of edited images by conditioning their editing to several reference views through self and cross-view attention between images’ latent representations. Experiments demonstrate that our method achieves faster editing and better visual results than previous state-of-the-art methods. Project website: https://gaussctrl.active.vision/
Citation
J. Wu et al., “GaussCtrl: Multi-view Consistent Text-Driven 3D Gaussian Splatting Editing,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , vol. 15072, pp. 55–71, Jan. 2025, doi: 10.1007/978-3-031-72630-9_4.
Source
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Conference
Keywords
3D Editing, Diffusion Models, Gaussian Splatting, Neural Radiance Fields
Subjects
Source
Publisher
Springer Nature