YoloS3DN: Towards Low-Latency ViT-Powered Stereo 3D Object Detection
Sebaitre, Zakaria
Sebaitre, Zakaria
Author
Supervisor
Department
Computer Vision
Embargo End Date
Type
Thesis
Date
2022
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Stereo 3D Object Detection has been an ever-growing challenge in the field of Computer Vision, specifically because of the role it plays in deploying Autonomous Driving solutions that are computationally light-weight, fast and accurate. This task is particularly challenging for models that utilise point cloud for their disparity estimation for depth 3D reconstruction. Moreover, Vision Transformers have recently started outperforming CNNs in image classification tasks, all whilst consuming fewer resources with fewer parameters and FLOPs. To this end, YoloS3DN is proposed, a light-weight, real-time capable iteration of YoloStereo3D with a Vision Transformer backbone. This network scales back into the 2D object detection whilst reinforcing them with stereo features. This model was trained on a single NVIDIA RTX 3090 GPU on the KITTI Stereo Dataset and validated on the 3D Object Detection Benchmark of 2017. Ablation and comparative experiments displaying experimental, comparative and qualitative results with superior performance in inference speed and state-of-the-art comparable performance in accuracy. The advancements in this field pushes innovation towards Stereo-based Object detection in real-time and real-world autonomous driving and Robotics solutions. The code is been made available at https://github.com/zakaseb/YoloS3DN.
Citation
Z. Sebaitre, "YoloS3DN: Towards Low-Latency ViT-Powered Stereo 3D Object Detection", M.S. Thesis, Computer Vision, MBZUAI, Abu Dhabi, UAE, 2022.