Jiahao Zheng
Author directory2026
ADRD-Bench: An LLM Benchmark for Alzheimer’s Disease and Related Dementias
Guangxin Zhao | Jiahao Zheng | Malaz Boustani | Jaroslaw Nabrzyski | Yiyu Shi | Meng Jiang | Zhi Zheng
Findings of the Association for Computational Linguistics: EMNLP 2026
Guangxin Zhao | Jiahao Zheng | Malaz Boustani | Jaroslaw Nabrzyski | Yiyu Shi | Meng Jiang | Zhi Zheng
Findings of the Association for Computational Linguistics: EMNLP 2026
Large language models (LLMs) have shown great potential for healthcare applications. However, existing evaluation benchmarks provide minimal coverage of Alzheimer’s Disease and Related Dementias (ADRD). To address this gap, we introduce ADRD-Bench, a preliminary ADRD-specific LLM benchmark. ADRD-Bench has two components: 1) ADRD Unified QA, a synthesis of 1,438 questions consolidated from seven established medical benchmarks, providing a unified assessment of clinical knowledge; and 2) ADRD Caregiving QA, a novel set of 149 questions derived from a nationally adopted, large clinical trials supported brain health management program, mitigating the lack of practical caregiving context in existing benchmarks. We evaluated 36 state-of-the-art LLMs on the proposed ADRD-Bench. Results showed that the accuracy of open-weight general models, open-weight medical models, and frontier closed-source general models ranged from 0.63 to 0.93 (mean: 0.77; std: 0.09), 0.47 to 0.93 (mean: 0.81; std: 0.14), and 0.83 to 0.93 (mean: 0.90; std: 0.03), respectively. While top-tier models achieved high accuracies (>0.9), case studies revealed inconsistent reasoning quality and stability, highlighting a critical need for domain-specific improvement to enhance LLMs’ knowledge and reasoning grounded in daily caregiving data. The ADRD-Bench dataset and code are publicly available at https://github.com/IIRL-ND/ADRD-Bench.