I Wrote This Because of That: Interpretable Radiology Report Generation via Visual Prototypes

Marco Salmè, Federico Siciliano, Fabrizio Silvestri, Paolo Soda, Rosa Sicilia, Valerio Guarrasi


Abstract
Automated Radiology Report Generation (RRG) could substantially reduce documentation workload by supporting physicians in routine clinical practice. Clinical adoption, however, requires each generated finding to be supported by inspectable evidence that a physician can verify. Existing systems either provide only post-hoc explanations weakly coupled to the prediction, or require costly region-level supervision to achieve spatial grounding. To address this gap, we introduce ProtoRRG, an RRG framework that grounds generation in visual prototypes. Specifically, each generated finding is tied to a dual form of evidence: localised image exemplars learned without anatomical supervision, and retrieved prior reports that supply case-based linguistic precedents. A multi-task training strategy reconciles the competing representational demands of prototype-based prediction and autoregressive decoding within a shared architecture. On MIMIC-CXR, ProtoRRG improves state-of-the-art clinical F1 and recall by 2.9% and 10%, respectively, with gains also demonstrated on IU X-Ray. An evaluation against anatomical annotations confirms that prototype activations localise within the correct regions, indicating spatially faithful evidence for the generated findings. These results demonstrate that combining visual prototypes with case-based retrieval improves both diagnostic accuracy and report quality while preserving the inspectable evidence that clinical deployment demands.
Anthology ID:
2026.findings-emnlp.1708
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
36511–36527
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1708/
DOI:
10.18653/2026.findings-emnlp.1708
Bibkey:
Cite (ACL):
Marco Salmè, Federico Siciliano, Fabrizio Silvestri, Paolo Soda, Rosa Sicilia, and Valerio Guarrasi. 2026. I Wrote This Because of That: Interpretable Radiology Report Generation via Visual Prototypes. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 36511–36527, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
I Wrote This Because of That: Interpretable Radiology Report Generation via Visual Prototypes (Salmè et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1708.pdf