Toward Robust Multi-Layered Embodied Robotic Decision Making Through Multilingual Reasoning and Uncertainty-Aware Grasping
Mansour, Malak Ibrahim Mohamed
Mansour, Malak Ibrahim Mohamed
Citations
Altmetric:
Supervisor
Department
Robotics
Embargo End Date
2027-05-01
Type
Thesis
Date
2026
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Embodied robotic intelligence requires reliable high-level reasoning and robust physical interaction under real-world uncertainty. Despite recent advances in large language models and robotic manipulation systems, current embodied agents often struggle with cross-lingual generalization in planning and with geometric uncertainty during physical interaction. This thesis investigates how linguistic evaluation and principled uncertainty modeling can improve robustness across multiple layers of embodied decision making.
At the high-level planning layer, we present the first multilingual evaluation of large and small language models for vision-and-language navigation within the NavGPT framework. Using the R2R dataset augmented with Arabic translations, we systematically assess the ability of state-of-the-art models to reason about navigation instructions across languages. Our results demonstrate that while modern models support structured planning in both English and Arabic, significant performance gaps emerge due to parsing limitations and uneven multilingual reasoning capabilities. These findings highlight the importance of robust linguistic grounding for embodied planning and reveal the potential of language-specific priors in robotics applications.
At the physical interaction layer, we address manipulation under severe occlusion through UNCLE-Grasp, an uncertainty-aware grasping pipeline for partially occluded strawberries. From a single partial observation, multiple incompatible 3D shape hypotheses may be plausible, making deterministic grasp selection unreliable. To address this challenge, we sample multiple shape completions via Monte Carlo dropout and aggregate grasp feasibility using a conservative lower confidence bound criterion. Experimental validation in both simulation and real-world robotic harvesting demonstrates improved robustness, safer abstention under high uncertainty, and superior grasp reliability compared to deterministic baselines.
Together, these contributions demonstrate that robustness in embodied robotic systems requires principled handling of both linguistic variability and geometric uncertainty. By systematically evaluating multilingual reasoning in navigation and uncertainty-aware decision making in manipulation, this thesis provides a unified perspective on designing more reliable embodied agents capable of operating under diverse real-world conditions.
Citation
Mansour, Malak Ibrahim Mohamed, "Toward Robust Multi-Layered Embodied Robotic Decision Making Through Multilingual Reasoning and Uncertainty-Aware Grasping," M.S. Thesis, Robotics, MBZUAI, 2026.
Source
Conference
Keywords
Robotic harvesting, Force closure, Occlusion handling, Uncertainty-aware grasping, Risk-aware decision making, Monte Carlo dropout