Cowpox: Towards the Immunity of VLM-based Multi-Agent Systems
Wu, Yutong Zhang, Jie Li, Yiming Zhang, Chao Guo, Qing QIU, Han Lukas, Nils Zhang, Tianwei
Wu, Yutong
Zhang, Jie
Li, Yiming
Zhang, Chao
Guo, Qing
QIU, Han
Lukas, Nils
Zhang, Tianwei
Supervisor
Department
Machine Learning
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Vision Language Model (VLM)-based agents are stateful, autonomous entities capable of perceiving and interacting with their environments through vision and language. Multi-agent systems comprise specialized agents who collaborate to solve a (complex) task. A core security property is robustness, stating that the system should maintain its integrity under adversarial attacks. However, the design of existing multi-agent systems lacks the robustness consideration, as a successful exploit against one agent can spread and infect other agents to undermine the entire system’s assurance. To address this, we propose a new defense approach, COWPOX, to provably enhance the robustness of multi-agent systems. It incorporates a distributed mechanism, which improves the recovery rate of agents by limiting the expected number of infections to other agents. The core idea is to generate and distribute a special cure sample that immunizes an agent against the attack before exposure and helps recover the already infected agents. We demonstrate the effectiveness of COWPOX empirically and provide theoretical robustness guarantees. The code can be found via https://github.com/WU-YU-TONG/Cowpox.
Citation
Y. Wu et al., “Cowpox: Towards the Immunity of VLM-based Multi-Agent Systems,” Oct. 06, 2025, PMLR. [Online]. Available: https://proceedings.mlr.press/v267/wu25aq.html
Source
Proceedings of Machine Learning Research
Conference
42nd International Conference on Machine Learning, ICML 2025
Keywords
Subjects
Source
42nd International Conference on Machine Learning, ICML 2025
Publisher
ML Research Press