Loading...
Multimodal Intention Recognition Combining Head Motion and Throat Vibration for Underwater Superlimbs
Zhang, Rongzheng Qiu, Wanghongjie Qiu, Jianuo Guo, Yuqin Dong, Chengxiao Zhang, Tuo Yi, Juan Song, Chaoyang Asada, Harry Wan, Fang
Zhang, Rongzheng
Qiu, Wanghongjie
Qiu, Jianuo
Guo, Yuqin
Dong, Chengxiao
Zhang, Tuo
Yi, Juan
Song, Chaoyang
Asada, Harry
Wan, Fang
Files
Loading...
2025.emnlp-main.1330.pdf
Adobe PDF, 5.49 MB
Supervisor
Department
Robotics
Embargo End Date
Type
Journal article
Date
License
http://creativecommons.org/licenses/by/4.0/
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
This paper presents a novel solution for underwater intention recognition that simultaneously detects head motion and throat vibration, enhancing multimodal human-robot interactions for underwater diving. The system pairs with an underwater supernumerary robotic limb (SuperLimb), providing propulsion assistance to reduce the diver’s physical load and mental fatigue. An inertial measurement unit monitors head motion, while a throat microphone captures vocal vibrations. Learning algorithms process these signals to accurately interpret the diver’s intentions and map them to the SuperLimb for posture management. The system features a compact design optimized for diving scenarios and includes a multimodal, real-time classification algorithm to distinguish various head motions and vocal signals. By collecting and analyzing underwater throat vibration data, the study demonstrates the feasibility of this approach, enabling continuous motion commands for enhanced diving assistance. The results show that the head motion recognition component of the system achieved a high classification accuracy of 95%, and throat vibration classification reached 86% accuracy on land and 89% underwater for various purposes. Note to Practitioners—This paper tackles the “diver’s wearable dilemma,” where effective communication and underwater robot control clash with traditional diver wearables’ constraints. Current methods, like hand gestures for interacting with underwater robots, disrupt natural limb movement and increase diver fatigue. To address this, we introduce a novel underwater intention recognition system that uses head movements and throat vibrations detected by an Inertial Measurement Unit (IMU) and a throat microphone. This hands-free interaction enhances wearable diving aids by reducing physical strain. We detail the algorithm for classifying and mapping these inputs to robot control commands, enabling real-time, continuous robot operation. Our approach significantly advances healthcare automation in commercial diving, improving diver safety and efficiency. Preliminary experiments show promise, but further field testing is needed to validate effectiveness in real-world conditions and ensure system reliability under extreme environments to prevent accidents from failures or interaction errors. Based on a user study conducted in Virtual Reality, the proposed interaction method maintains a consistent mental workload for users across tasks of varying difficulty and does not interfere with other human functions.
Citation
R. Zhang, W. Qiu, J. Qiu, Y. Guo, C. Dong, T. Zhang, J. Yi, C. Song, H. Asada, F. Wan, "Multimodal Intention Recognition Combining Head Motion and Throat Vibration for Underwater Superlimbs," IEEE Transactions on Automation Science and Engineering, vol. 23, pp. 2268-2281, 2026, https://doi.org/10.1109/tase.2025.3554036.
Source
IEEE Transactions on Automation Science and Engineering
Conference
Keywords
40 Engineering, 4007 Control Engineering, Mechatronics and Robotics
Subjects
Source
Publisher
IEEE