Fra-ViTac: Visuotactile Grasping for Delicate Strawberry Harvesting
Zhou, Muxin
Zhou, Muxin
Author
Supervisor
Department
Machine Learning
Embargo End Date
2026-05-30
Type
Thesis
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Compared to other crops and fruits, strawberries are highly fragile and susceptible to mechanical damage during harvesting. To minimize such damage and enhance harvesting performance, we propose Fra-ViTac, a novel diffusion model that integrates visual information from a camera and tactile data from a customdesigned tactile sensor, enabling precise control of the gripper force during picking. Our algorithm is implemented in a strawberry-specific harvesting system, utilizing a SIM(3)-equivariant architecture to enhance robustness in handling strawberries with varying poses and scales. Additionally, we employ a 6D continuous representation for rotations, which accelerates network convergence. Furthermore, we introduce a Transformerbased noise prediction model within the diffusion framework, which enhances the success rate from the baseline 68% to 100%. We also propose a novel loss function tailored to optimize the diffusion model for strawberry harvesting. The loss function approach achieves a 28% increase in success rate compared to the baseline, demonstrating the superiority of our algorithm.
Citation
Muxin Zhou, “Fra-ViTac: Visuotactile Grasping for Delicate Strawberry Harvesting,” Master of Science thesis, Machine Learning, MBZUAI, 2025.
Source
Conference
Keywords
Robotic Tactile Perception, Diffusion Policy, Robotic Automatic Harvesting