Multi-Scale Articulated Human Reconstruction by Adaptive Optimization on Sparse Images
Fei, Yulang
Fei, Yulang
Author
Supervisor
Department
Machine Learning
Embargo End Date
2024-01-01
Type
Thesis
Date
2024
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Recently, 3D Gaussian Splatting has emerged as a highly effective technique for novel view synthesis, particularly in reconstructing human models from monocular videos. This method surpasses traditional NeRF-based implicit representation models in both efficiency and visual fidelity. However, its reliance on an extensive amount of Gaussians introduces challenges such as increased memory consumption, longer training periods, and the potential for noticeable artifacts when adjusting the sampling rate. To mitigate these issues, we have devised an innovative approach that uses JensenShannon divergence to dynamically adjust Gaussian points. This technique efficiently controls the Gaussian count to around 7,000 per person, significantly streamlining the optimization process without compromising quality. Moreover, we employ a refined training strategy that involves rendering images at various resolutions and segmenting the original image into 16 sub-images. This segmentation enhances performance by improving visual fidelity, enabling the comprehensive reconstruction of a person with just 20 images. Additionally, the integration of a 3D filter helps in smoothing out any resulting artifacts, further enhancing the overall quality of the reconstructions.
Citation
Y. Fei, "Multi-Scale Articulated Human Reconstruction by Adaptive Optimization on Sparse Images", MS. Thesis, Machine Learning, MBZUAI, Abu Dhabi, UAE, 2024