Siran Zhao
Author directory2026
MedCalc-Pro: Solving Complex Medical Calculations with LLM Agents
Siran Zhao | Ruihui Hou | Ziyue Huai | Chennuo Zhang | Tong Ruan
Findings of the Association for Computational Linguistics: EMNLP 2026
Siran Zhao | Ruihui Hou | Ziyue Huai | Chennuo Zhang | Tong Ruan
Findings of the Association for Computational Linguistics: EMNLP 2026
Current benchmarks for evaluating large language models (LLMs) in medical calculation are largely based on simplified settings, where each patient case corresponds to a single calculator and the required tool is explicitly specified in the query. However, real clinical scenarios often require multiple calculators for joint evaluation, nested-calculator calculation, and fuzzy queries that do not directly specify the target calculator. To this end, we propose a new medical calculation benchmark, MedCalc-Pro, which covers three progressively challenging task settings: single-calculator, multi-calculator, and nested-calculator calculation settings. MedCalc-Pro contains 2,268 real-world clinical cases, covering 77 medical calculators across 14 clinical departments. Meanwhile, to address the limited performance of existing frameworks and methods in complex clinical scenarios, we further propose a more generalizable agent framework that supports multi-calculator selection and nested-calculator calling, while suppressing parameter error propagation through structured validation and evidence audit. We conduct systematic comparisons across open-source, closed-source, and medical-specialized LLMs, and the results show that our framework achieves the best performance across all three task settings. This work provides a new benchmark and method for evaluating and applying LLMs in challenging medical calculation scenarios. The dataset and code are publicly available at https://github.com/easymoney7774/MedCalc-Pro.