MALMM: Multi-Agent Large Language Models for Zero-Shot Robotic Manipulation
Singh, Harsh Das, Rocktim Jyoti Han, Mingfei Nakov, Preslav Laptev, Ivan
Singh, Harsh
Das, Rocktim Jyoti
Han, Mingfei
Nakov, Preslav
Laptev, Ivan
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Conference proceeding
Date
License
Language
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Large Language Models (LLMs) have demonstrated remarkable planning abilities across various domains, including robotic manipulation and navigation. While recent work in robotics deploys LLMs for high-level and low-level planning, existing methods often face challenges with failure recovery and suffer from hallucinations in long-horizon tasks. To address these limitations, we propose a novel multi-agent LLM framework, Multi-Agent Large Language Model for Manipulation (MALMM). Notably, MALMM distributes planning across three specialized LLM agents, namely high-level planning agent, low-level control agent, and a supervisor agent. Moreover, by incorporating environment observations after each step, our framework effectively handles intermediate failures and enables adaptive re-planning. Unlike existing methods, MALMM does not rely on pre-trained skill policies or in-context learning examples and generalizes to unseen tasks. In our experiments, MALMM demonstrates excellent performance in solving previously unseen long-horizon manipulation tasks, and outperforms existing zero-shot LLM-based methods in RLBench by a large margin. Experiments with the Franka robot arm further validate our approach in real-world settings.
Citation
H. Singh, R.J. Das, M. Han, P. Nakov, I. Laptev, "MALMM: Multi-Agent Large Language Models for Zero-Shot Robotic Manipulation," 2025, pp. 20386-20393.
Source
2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Conference
2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Keywords
46 Information and Computing Sciences, 4602 Artificial Intelligence
Subjects
Source
2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Publisher
IEEE