MedAgentSim: Self-evolving Multi-agent Simulations for Realistic Clinical Interactions
Almansoori, Mohammad Kumar, Komal Cholakkal, Hisham
Almansoori, Mohammad
Kumar, Komal
Cholakkal, Hisham
Author
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
In this work, we introduce MedAgentSim, an open-source simulated clinical environment with doctor, patient, and measurement agents designed to evaluate and enhance LLM performance in dynamic diagnostic settings. Unlike prior approaches, our framework requires doctor agents to actively engage with patients through multi-turn conversations, requesting relevant medical examinations (e.g., temperature, blood pressure, ECG) and imaging results (e.g., MRI, X-ray) from a measurement agent to mimic the real-world diagnostic process. Additionally, we incorporate self improvement mechanisms that allow models to iteratively refine their diagnostic strategies. We enhance LLM performance in our simulated setting by integrating multi-agent discussions, chain-of-thought reasoning, and experience-based knowledge retrieval, facilitating progressive learning as doctor agents interact with more patients. We also introduce an evaluation benchmark for assessing the LLM’s ability to engage in dynamic, context-aware diagnostic interactions. While MedAgentSim is fully automated, it also supports a user-controlled mode, enabling human interaction with either the doctor or patient agent. Comprehensive evaluations in various simulated diagnostic scenarios demonstrate the effectiveness of our approach. Our codebase, simulation environment, and benchmark datasets are publicly available on the project page: https://medagentsim.netlify.app/
Citation
M. Almansoori, K. Kumar, and H. Cholakkal, “MedAgentSim: Self-evolving Multi-agent Simulations for Realistic Clinical Interactions,” pp. 362–372, 2026, doi: 10.1007/978-3-032-05114-1_35
Source
Medical Image Computing and Computer Assisted Intervention
Conference
MICCAI 2025- Medical Image Computing and Computer Assisted Intervention
Keywords
Multi Agents, Visual Agents, Self Improving Agents
Subjects
Source
MICCAI 2025- Medical Image Computing and Computer Assisted Intervention
Publisher
Springer Nature