A Virtual Try-On System for Any Clothing and Body Model
Cao, Cong
Cao, Cong
Author
Cao, Cong
Supervisor
Department
Computer Vision
Embargo End Date
2025-05-21
Type
Thesis
Date
2024
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Virtual try-on (VTON) technology is a powerful tool for e-commerce, fashion design, and digital content creation. However, previous work, like physical-based simulation, although it provides realistic results, requires expertise and manual adjustment and can not achieve real time performance. In this study, we propose to develop a deep learning-based cloth simulation framework that can quickly fit any clothing model onto any body model, regardless of whether the body model is parametric or non-parametric. Our system not only automates the previously labor-intensive process of fitting garments to varied body shapes and poses, but also introduces flexibility and customization capabilities in garment sizing and styling. Our system is composed of registration module, cloth transfer module, and collisionsolving module. Firstly, we render the original body model in multiple views, detect keypoints and joints from the 2D images, and lift 2D pose to 3D as a nice initialization. Thus, the SMPL with the initialized pose is optimized until the shape and pose of the SMPL align with the original model. Secondly, the network integrates the loss functions that formed from physical principles, thus governing cloth movement and deformation in a realistic way and realizing self-supervised training. The inference pipeline is implemented to realize smooth garment mesh adaptation across diverse body types. Thirdly, the collision solving module is used to produce a collision-free result. The experiment results prove the system s effectiveness, showcasing its ability to accurately dress clothing on different body types and efficiently manage challenging situations like collisions and self-collisions. The enhanced style customization brings more possibilities for creative users, makes our work a reliable and adaptable tool for the fashion industry and other fields.
Citation
C.Cao, "A Virtual Try-On System for Any Clothing and Body Model", M.S. Thesis, Computer Vision, MBZUAI, Abu Dhabi, UAE, 2024