Real‐Time Neural Materials on Mobile VR
Xu, Zilin Zhou, Yang Litman, Yehonathan Chiang, Matt Jen‐Yuan Yan, Lingqi Michels, Anton
Xu, Zilin
Zhou, Yang
Litman, Yehonathan
Chiang, Matt Jen‐Yuan
Yan, Lingqi
Michels, Anton
Supervisor
Department
Computer Science
Embargo End Date
Type
Journal article
Date
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Abstract Virtual Reality (VR) applications aim to create an immersive virtual world, which demands a high level of visual realism. The analytical material models commonly used in VR often fall short of reproducing complex real‐world appearances. Recently, neural materials have emerged as a promising alternative, offering a compact yet effective representation of real‐world materials. Deploying neural materials on low‐power mobile VR devices poses significant challenges due to the computational complexity of neural networks and the high display resolution and frame rate requirements of VR devices (commonly 72+ frames per second). We address these challenges by leveraging texture‐space shading with spatiotemporal computation amortization, driven by a compact, coarse‐to‐fine neural material model of extremely low capacity. Thanks to our distillation training scheme, our compact neural materials achieve visual quality comparable to NeuMIP [KMX*21] at a much lower cost. Our method reaches over 90 FPS on a mobile VR device (Meta Quest 3) even under multiple light sources.
Citation
Z. Xu, Y. Zhou, Y. Litman, M.J. Chiang, L. Yan, A. Michels, "Real‐Time Neural Materials on Mobile VR," Computer Graphics Forum, 2026, https://doi.org/10.1111/cgf.70318.
Source
Computer Graphics Forum
Conference
Keywords
46 Information and Computing Sciences, 4607 Graphics, Augmented Reality and Games
Subjects
Source
Publisher
Wiley