Improving Identity Consistency for Neural Head Reeanactment
Hoang, Steven Phong
Hoang, Steven Phong
Author
Supervisor
Department
Machine Learning
Embargo End Date
Type
Thesis
Date
2023
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
"Neural reenactment systems are becoming increasingly popular for generating realistic human portraits by extracting the pose and expression of a driver image and transferring it to a source image. However, current solutions struggle with identity change and leakage, and can result in inconsistent and undesirable facial expressions. In this thesis, we propose an approach that disentangles driver and source identities to improve identity consistency while maintaining high-fidelity results. Our approach utilizes multi-frame support, sparsity in expression latent vectors, and high-quality datasets for training. We also introduce a super-resolution module to further enhance our results. Our contributions include a neural reenactment model that improves identity consistency, techniques for encouraging disentanglement, curation of high-quality reenactment datasets, and a super-resolution module."
Citation
S.P. Hoang, "Improving Identity Consistency for Neural Head Reeanactment", M.S. Thesis, Machine Learning, MBZUAI, Abu Dhabi, UAE, 2023.