An Agentic AI Framework for In-Vehicle Safety System
Almarzooqi, Abdulrahman Hasan
Almarzooqi, Abdulrahman Hasan
Supervisor
Department
Machine Learning
Embargo End Date
2025-05-30
Type
Thesis
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
The increasing number of traffic accidents, many of which are directly related to driver behavior, has underscored the urgent need for effective and safe invehicle monitoring systems. In response to this critical safety challenge, our study introduces a collaborative agentic artificial intelligence (AI) framework designed to provide comprehensive assessments of key invehicle safety parameters. This framework integrates six distinct monitoring components: driver state, seatbelt usage, phone usage, vehicle speed, presence of alcohol, and child occupancy and distraction. Each component is managed by a specialized AI agent that implements a multimodal large language model (MLLM), specifically GPT4o mini, and is responsible for evaluating its assigned parameter. The system processes inputs dynamically through configuration files, with each agent analyzing images or numerical data relevant to its task. The agents’ output is processed sequentially and aggregated into a cohesive safety report, offering a complete assessment of the vehicle’s safety status. The unified framework was evaluated using 30 different image sets, each of which contained data corresponding to the six monitored parameters. Experimental results revealed the framework’s high accuracy in detecting and classifying the critical safety indicators, with an average overall safety score of 95.0%. These findings validate the effectiveness of the framework, highlighting its potential to improve road safety by providing an integrated, intelligent invehicle monitoring solution. As part of developing the driver state monitoring component, we additionally conducted a comparative analysis to explore the capabilities of various stateoftheart AI models for driver state classification. Although separate from the unified framework, this evaluation provided insight into the broader landscape of driver monitoring solutions. Specifically, we compared the performance of GPT-4o, finetuned quantized Llama 3.2 11B Vision Instruct, and a finetuned vision transformer (ViT)large. These models were tested on a dataset of driver state images categorized into alert and drowsy classes. Performance was measured using accuracy, precision, recall, and the F1 score. The results indicated that the finetuned large ViT achieved the best classification accuracy, demonstrating its strong potential for driver state classification tasks.
Citation
Abdulrahman Hasan Almarzooqi, “An Agentic AI Framework for In-Vehicle Safety System,” Master of Science thesis, Machine Learning, MBZUAI, 2025.
Source
Conference
Keywords
Agentic, In-vehicle, ViT, Llama 3.2, GPT-4o