Towards Efficient Neural Radiance Field For Real-world Applications
Bagudu, Aliyu
Bagudu, Aliyu
Author
Supervisor
Department
Computer Vision
Embargo End Date
2024-05-21
Type
Thesis
Date
2023
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Neural Radiance Field (NeRF) is state-of-the-art computer vision technology that has tremendious applications and opportunity of improvements. NeRF is built upon funda- mentals of computer vision (pinhole camera model) and therefore provides an opportunity for a wide range of applications that can be built off of it. To clarify, computer vision is a primary branch of artificial intelligence (AI). Thus, NeRF has wide range of applications that makes research in it undoubtedly worthwhile. It applications include but not limited to generation 3D models of objects, modeling of virtual and augmented reality scenes in the metaverse, rendering scenes for 3D video games, rendering and persisting of medical imagery data, such as magnetic resonance imaging (MRI) and ultrasound scans. Speaking generally, NeRF is applicable to processing (interpolation and efficient persistence) of any type of signals(i.e., ariel satellite images, time-series-based). However, NeRF requires further research, as the technique does face limitations. Specif- ically, this technique currently needs improvements in composability, time-sensitivity, and editability. Thus, the research on optimizing NeRF is highly valuable in light of the vast, real- world applications that the optimization of NeRF stands to benefit. Effiecient NeRF solutions are proposed. This results from a novel, recursive, value based, volume rendering model proposed. This feat is accomplished through an application of a Computational Science & Engineering (CSE)centric approach to NeRF, whereby the NeRF model is viewed from a CSE lens. This provides better understanding of the problem and solution algorithms and thereby enables proposal of suitable modifications. This work reduces NeRF solution to a Reinforced Learning (RL) solution with massive implication that NeRF solutions can benefit from RL solutions and vice versa. An abstract advantage Reinforced NeRF gains from RL is that a real-time value based solution is fea- sible without caching (or baking) a pre-trained model. On the other hand, NeRF solution is able to predict and represent a complex high dimensional signal in great resolution and accuracy from a sparse lower dimensional training samples. Therefore RL solutions stand to benefit from this. JAXNeRFnoRays is proposed as by product of efforts on Reinforced NeRF. This is a single shot prediction of ray values. JAXNeRFnoRays out performs NeRF and JAXNeRF by order of hundreds magnitude in inference speed. It also has a better space performance due to the use of a single MLP model.
Citation
A. Bagudu, "Towards Efficient Models for Continual Learning in Medical Imaging", M.S. Thesis, Computer Vision, MBZUAI, Abu Dhabi, UAE, 2023