Loading...
AniFaceDiff: Animating stylized avatars via parametric conditioned diffusion models
Chen, Ken Seneviratne, Sachith Wang, Wei Hu, Dongting Saha, Sanjay Hasan, Md. Tarek Rasnayaka, Sanka Malepathirana, Tamasha Gong, Mingming Halgamuge, Saman
Chen, Ken
Seneviratne, Sachith
Wang, Wei
Hu, Dongting
Saha, Sanjay
Hasan, Md. Tarek
Rasnayaka, Sanka
Malepathirana, Tamasha
Gong, Mingming
Halgamuge, Saman
Supervisor
Department
Machine Learning
Embargo End Date
Type
Journal article
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Animating stylized head avatars with dynamic poses and expressions has become an important focus in recent research due to its broad range of applications (e.g. VR/AR, film and animation, privacy protection). Previous research has made significant progress by training controllable generative models to animate the reference avatar using the target pose and expression. However, existing portrait animation methods are mostly trained using human faces, making them struggle to generalize to stylized avatar references such as cartoon, painting, and 3D-rendered avatars. Moreover, the mechanisms used to animate avatars – namely, to control the pose and expression of the reference – often inadvertently introduce unintended features – such as facial shape – from the target, while also causing a loss of intended features, like expression-related details. This paper proposes AniFaceDiff, a Stable Diffusion based method with a new conditioning module for animating stylized avatars. First, we propose a refined spatial conditioning approach by Facial Alignment to minimize identity mismatches, particularly between stylized avatars and human faces. Then, we introduce an Expression Adapter that incorporates additional cross-attention layers to address the potential loss of expression-related information. Extensive experiments demonstrate that our method achieves state-of-the-art performance, particularly in the most challenging out-of-domain stylized avatar animation, i.e., domains unseen during training. It delivers superior image quality, identity preservation, and expression accuracy. This work enhances the quality of virtual stylized avatar animation for constructive and responsible applications. To promote ethical use in virtual environments, we contribute to the advancement of face manipulation detection by evaluating state-of-the-art detectors, highlighting potential areas for improvement, and suggesting solutions.
Citation
“AniFaceDiff: Animating stylized avatars via parametric conditioned diffusion models,” Pattern Recognit, vol. 170, p. 112017, Feb. 2026, doi: 10.1016/J.PATCOG.2025.112017
Source
Pattern Recognition
Conference
Keywords
Subjects
Source
Publisher
Elsevier