Burst Image Restoration and Enhancement
Dudhane, Akshay Zamir, Syed Waqas Khan, Salman Khan, Fahad Shahbaz Yang, Ming-Husan
Dudhane, Akshay
Zamir, Syed Waqas
Khan, Salman
Khan, Fahad Shahbaz
Yang, Ming-Husan
Supervisor
Department
Computer Vision
Embargo End Date
Type
Journal article
Date
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Burst Image Restoration aims to reconstruct a high-quality image by efficiently combining complementary inter-frame information. However, it is quite challenging since individual burst images often have inter-frame misalignments that usually lead to ghosting and zipper artifacts. To mitigate this, we develop a novel approach for burst image processing named BIPNet that focuses solely on the information exchange between burst frames and filter-out the inherent degradations while preserving and enhancing the actual scene details. Our central idea is to generate a set of pseudo-burst features that combine complementary information from all the burst frames to exchange information seamlessly. However, due to inter-frame misalignment, the information cannot be effectively combined in pseudo-burst. Thus, we initially align the incoming burst features regarding the reference frame using the proposed edge-boosting feature alignment. Lastly, we progressively upscale the pseudo-burst features in multiple stages while adaptively combining the complementary information. Unlike the existing works, that usually deploy single-stage up-sampling with a late fusion scheme, we first deploy a pseudo-burst mechanism followed by the adaptive-progressive feature up-sampling. The proposed BIPNet significantly outperforms the existing methods on burst super-resolution, low-light image enhancement, low-light image super-resolution, and denoising tasks.
Citation
A. Dudhane, S.W. Zamir, S. Khan, F.S. Khan, M.-H. Yang, "Burst Image Restoration and Enhancement," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 11, pp. 9454-9467, 2024, https://doi.org/10.1109/TPAMI.2024.3356188.
Source
IEEE Transactions on Pattern Analysis and Machine Intelligence
Conference
Keywords
46 Information and Computing Sciences, 4603 Computer Vision and Multimedia Computation
Subjects
Source
Publisher
IEEE