When Glass Disappears at Night: A Novel NIR-RGB Multimodal Solution
Yan, Tao Lu, Yiwei Xu, Ke Chen, Hao Li, Hui Chang, Xiaojun Wu, Xiaojun Lau, Rynson
Yan, Tao
Lu, Yiwei
Xu, Ke
Chen, Hao
Li, Hui
Chang, Xiaojun
Wu, Xiaojun
Lau, Rynson
Citations
Altmetric:
Supervisor
Department
Computer Vision
Embargo End Date
Type
Journal article
Date
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Glass surface detection (GSD) has recently been attracting research interests. However, existing GSD methods focus on modeling glass surface properties for daytime scenes only, and can easily fail in nighttime scenes due to significant lighting discrepancies. We observe that, due to the spectral differences between Near-Infrared (NIR) light sources and common LED lights, NIR and RGB cameras capture complementary visual patterns (e.g., light reflections, shadows, and edges) of glass surfaces, and cross-comparing their lighting and reflectance properties can provide reliable cues for nighttime GSD. Inspired by this observation, we propose a novel approach for nighttime GSD based on the multi-modal NIR and RGB image pairs. We first construct a nighttime GSD dataset, which contains 6, 192 RGB-NIR image pairs captured in diverse real-world nighttime scenes, with corresponding carefully-annotated glass surface masks. We then propose a novel network for the nighttime GSD task with two novel modules: (1) an RGB-NIR Guidance Enhancement (RNGE) module for extracting and enriching the NIR reflectance features with the guidance of RGB reflectance features, and (2) an RGB-NIR Fusion and Localization (RNFL) module for fusing RGB and NIR reflectance features into glass features conditioned on the multi-modal illumination discrepancy-aware features. Extensive experiments demonstrate that our method outperforms state-of-the-art methods in nighttime scenes while generalizing well to daytime scenes. Our dataset and code are available at https://github.com/YT3DVision/NGSDNet.
Citation
T. Yan, Y. Lu, K. Xu, H. Chen, H. Li, X. Chang , et al., "When Glass Disappears at Night: A Novel NIR-RGB Multimodal Solution," Transactions on Machine Learning Research, vol. 2026-May, 2026,
Source
Transactions on Machine Learning Research