Xiangyang Zhu
Author directory2026
MMedAgentBench: A Multi-Stage Benchmark for Evaluating LLM-based Medical Agents
Lu Sun | Xiangyi Zhang | Xiangyang Zhu | Kaiyuan Ji | Zijian Chen | Qiang Hu | Zicheng Zhang | Yuan Tian | Guangtao Zhai
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Lu Sun | Xiangyi Zhang | Xiangyang Zhu | Kaiyuan Ji | Zijian Chen | Qiang Hu | Zicheng Zhang | Yuan Tian | Guangtao Zhai
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models (LLMs) have demonstrated remarkable potential in medical AI through their natural language understanding and reasoning capabilities. However, real clinical practice involves multi-stage, dynamically evolving decision-making where physicians continuously adjust pathways based on updated examination results—far more complex than single-turn question answering. Existing medical benchmarks focus on static scenarios, failing to evaluate LLMs in realistic clinical environments. We present MMedAgentBench, a comprehensive benchmark for evaluating LLM-based medical agents in multi-stage clinical decision-making. Unlike previous benchmarks, MMedAgentBench captures the temporal complexity of clinical workflows by simulating authentic patient journeys across multiple diagnostic and treatment stages. The benchmark comprises a high-quality dataset of over 150 de-identified clinical cases from tertiary hospitals, encompassing 500+ clinical stages, with each case meticulously structured into sequential stages and annotated by board-certified physicians. Our evaluation framework simultaneously assesses diagnostic accuracy and resource efficiency, requiring agents to plan examinations and control costs while navigating progressive information disclosure. This enables systematic evaluation of medical agents across dynamic decision-making, cost control, and multi-stage reasoning dimensions, providing a realistic testbed for advancing medical AI toward clinical deployment.
One Battle After Another: Probing LLMs’ Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework
Qi Jia | Ye Shen | Xiujie Song | Kaiwei Zhang | Shibo Wang | Dun Pei | Xiangyang Zhu | Guangtao Zhai
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Qi Jia | Ye Shen | Xiujie Song | Kaiwei Zhang | Shibo Wang | Dun Pei | Xiangyang Zhu | Guangtao Zhai
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Evaluating LLMs’ instruction-following ability in multi-topic dialogues is essential yet challenging. Existing benchmarks are limited to a fixed number of turns, susceptible to saturation and failing to account for users’ interactive experience. In this work, we propose a novel framework featuring a three-layer tracking mechanism and a query synthesis agent to mimic sequential user behaviors. Grounded in Flow Theory, we introduce process-centric metrics and terminate a conversational evaluation only upon exhausting user patience. Leveraging this framework, we present EvolIF, an evolving benchmark covering 12 constraint groups. Our analysis reveals deficiencies in failure recovery and fine-grained instruction following, with performance stratification becoming evident as conversational depth increases. GPT-5 demonstrates the most sustained resilience, maintaining a 66.40% stability score, outperforming Gemini-3-Pro by 5.59%, while other models lag behind.