A Practical Multi-View System for Building Neural Head Avatars
Lindsay, Kane
Lindsay, Kane
Author
Supervisor
Department
Computer Vision
Embargo End Date
2024-01-01
Type
Thesis
Date
2024
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Digital human head avatars show promise in applications, including the expanding metaverse, digital telepresence, entertainment, and educational tools. However, the traditional methods, such as manual modelling and photogrammetry, to create such models are time-consuming, laborious or prone to human error. Current automated processes such as Light Stage are prohibitively expensive and require highly specialised engineering to build, so they are not widely available. This work explores the feasibility of deep-learning neural methods to represent four-dimensional head avatars captured in 360 degrees with the addition of dynamic movements, allowing for the digital recreation of any real human head given only sparse calibrated images. An end-to-end pipeline is proposed and executed from hardware set-up to avatar rendering, including the creation of a sparse-view dataset using a non-rigid system such that camera positions and orientations can be adjusted for testing different configurations. It is found that currently available state-of-the-art methods focusing on frontal facial capture using multi-resolution spatial hashing representations may be extended to a 360-degree sparse setting while retaining reasonable quality, suggesting directions for the future development of head avatar capturing systems.
Citation
K. Lindsay, "A Practical Multi-View System for Building Neural Head Avatars", M.S. Thesis, Computer Vision, MBZUAI, Abu Dhabi, UAE, 2024