Iterative Shaping of Multi-Particle Aggregates Based on Action Trees and VLM
Lee, Hoi-Yin Zhou, Peng Duan, Anqing Yang, Chenguang Navarro-Alarcon, David
Lee, Hoi-Yin
Zhou, Peng
Duan, Anqing
Yang, Chenguang
Navarro-Alarcon, David
Supervisor
Department
Robotics
Embargo End Date
Type
Journal article
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
In this paper, we address the problem of manipulating multi- particle aggregates using a bimanual robotic system. Our approach enables the autonomous transport of dispersed particles through a series of shaping and pushing actions using robotically controlled tools. Achieving this advanced manipulation capability presents two key challenges: high-level task planning and trajectory execution. For task planning, we leverage Vision Language Models (VLMs) to enable primitive actions such as tool affordance grasping and non-prehensile particle pushing. For trajectory execution, we represent the evolving particle aggregate's contour using truncated Fourier series, providing efficient parametrization of its closed shape. We adaptively compute trajectory waypoints based on group cohesion and the geometric centroid of the aggregate, accounting for its spatial distribution and collective motion. Through real-world experiments, we demonstrate the effectiveness of our methodology in actively shaping and manipulating multi- particle aggregates while maintaining high system cohesion.
Citation
H. -Y. Lee, P. Zhou, A. Duan, C. Yang and D. Navarro-Alarcon, "Iterative Shaping of Multi-Particle Aggregates Based on Action Trees and VLM," in IEEE Robotics and Automation Letters, doi: 10.1109/LRA.2025.3572426
Source
IEEE Robotics and Automation Letters
Conference
Keywords
Robot manipulation, Shape control, Action trees, Multi-particle manipulation, VLM
Subjects
Source
Publisher
IEEE