The Diagnosis Is Already in the Notes: A Channel-Wise Audit of Diagnostic Shortcuts in Clinical-Text Benchmarks

Haocun Ye, Xinlong Jiang, Yuanyuan Hu, Shubai Chen, Tingyu Wu, Zhenkun Zheng, Caijiao Yi, Teng Zhang, Liuxin Wang, Tong Zhao, Heping Xu, Weiwei Dai, Yiqiang Chen


Abstract
Medical AI benchmarks routinely pair a diagnostic image with the same-visit clinical record, delete the recorded diagnosis, and treat what remains as auxiliary context. The recipe is reasonable. It is not. On 5904 ophthalmic visits across eight cohorts, a frozen text probe still recovers the disease label from the diagnosis-redacted record at 0.944 macro-AUROC, against 0.998 on the unredacted record; the most aggressive dictionary scrub closes less than one further percentage point. The diagnosis is already in the notes.

It is not in one place. A clinical record is written during the diagnostic process, and its fingerprints persist across channels: narrative text alone reaches 0.978, and a view that erases every value and exposes only which fields were populated still reaches 0.854. We introduce CRAFT (Channel-wise Redaction And Floor Test), a portable audit protocol that renders each record into redacted and channel-isolated views and reports each channel’s label recoverability; rankings reproduce under Qwen3.5-9B. Before clinical-text or image-text fusion gains are read as cross-modal reasoning, the text branch’s channel-level shortcut floors must be reported alongside them. https://github.com/yhc98002-bit/The-Diagnosis-Is-Already-in-the-Notes
Anthology ID:
2026.findings-emnlp.381
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
8041–8060
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.381/
DOI:
Bibkey:
Cite (ACL):
Haocun Ye, Xinlong Jiang, Yuanyuan Hu, Shubai Chen, Tingyu Wu, Zhenkun Zheng, Caijiao Yi, Teng Zhang, Liuxin Wang, Tong Zhao, Heping Xu, Weiwei Dai, and Yiqiang Chen. 2026. The Diagnosis Is Already in the Notes: A Channel-Wise Audit of Diagnostic Shortcuts in Clinical-Text Benchmarks. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 8041–8060, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
The Diagnosis Is Already in the Notes: A Channel-Wise Audit of Diagnostic Shortcuts in Clinical-Text Benchmarks (Ye et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.381.pdf