Xiangyi Zhang
Author directory2026
MMedAgentBench: A Multi-Stage Benchmark for Evaluating LLM-based Medical Agents
Lu Sun | Xiangyi Zhang | Xiangyang Zhu | Kaiyuan Ji | Zijian Chen | Qiang Hu | Zicheng Zhang | Yuan Tian | Guangtao Zhai
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Lu Sun | Xiangyi Zhang | Xiangyang Zhu | Kaiyuan Ji | Zijian Chen | Qiang Hu | Zicheng Zhang | Yuan Tian | Guangtao Zhai
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models (LLMs) have demonstrated remarkable potential in medical AI through their natural language understanding and reasoning capabilities. However, real clinical practice involves multi-stage, dynamically evolving decision-making where physicians continuously adjust pathways based on updated examination results—far more complex than single-turn question answering. Existing medical benchmarks focus on static scenarios, failing to evaluate LLMs in realistic clinical environments. We present MMedAgentBench, a comprehensive benchmark for evaluating LLM-based medical agents in multi-stage clinical decision-making. Unlike previous benchmarks, MMedAgentBench captures the temporal complexity of clinical workflows by simulating authentic patient journeys across multiple diagnostic and treatment stages. The benchmark comprises a high-quality dataset of over 150 de-identified clinical cases from tertiary hospitals, encompassing 500+ clinical stages, with each case meticulously structured into sequential stages and annotated by board-certified physicians. Our evaluation framework simultaneously assesses diagnostic accuracy and resource efficiency, requiring agents to plan examinations and control costs while navigating progressive information disclosure. This enables systematic evaluation of medical agents across dynamic decision-making, cost control, and multi-stage reasoning dimensions, providing a realistic testbed for advancing medical AI toward clinical deployment.