MedInsightBench: Evaluating Medical Analytics Agents Through Multi-Step Insight Discovery in Multi-modal Medical Data

Zhenghao Zhu, Chuxue Cao, Yuanfeng Song, Xing Chen, Caleb Chen Cao, Sirui Han, Yike Guo


Abstract
In medical data analysis, extracting deep insights from complex multi-modal pathological data is essential for enhancing patient care, diagnostic accuracy, and healthcare operations. Despite this clinical significance, current benchmarks fail to adequately evaluate Large Multi-modal Models (LMMs) on medical insight discovery. Therefore, we introduce MedInsightBench, the first benchmark comprising 332 curated cases from The Cancer Genome Atlas, each annotated with expert-derived insights. Instead of simple QA, this benchmark evaluates whether LMMs can derive clinically grounded insights tied to tumor staging, prognosis, and biomarker expression. Our initial analysis reveals that existing LMMs struggle with MedInsightBench, primarily due to a lack of domain expertise and an inability to perform multi-step reasoning. Consequently, we propose MedInsightAgent, an automated agent framework featuring three specialized modules: Visual Root Finder, Analytical Insight Agent, and Follow-up Question Composer. Extensive experiments confirm that MedInsightAgent effectively addresses the limitations of existing models, consistently demonstrating superior performance in discovering complex medical insights.
Anthology ID:
2026.emnlp-main.1272
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
28984–29016
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1272/
DOI:
Bibkey:
Cite (ACL):
Zhenghao Zhu, Chuxue Cao, Yuanfeng Song, Xing Chen, Caleb Chen Cao, Sirui Han, and Yike Guo. 2026. MedInsightBench: Evaluating Medical Analytics Agents Through Multi-Step Insight Discovery in Multi-modal Medical Data. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 28984–29016, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
MedInsightBench: Evaluating Medical Analytics Agents Through Multi-Step Insight Discovery in Multi-modal Medical Data (Zhu et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1272.pdf