The More, The Better? A Critical Study of Multimodal Context in Radiology Report Summarization

Mong Yuan Sim, Wei Emma Zhang, Xiang Dai, Biaoyan Fang, Sarbin Ranjitkar, Arjun Burlakoti, Jamie Taylor, Haojie Zhuang


Abstract
The Impression section of a radiology report summarizes critical findings of a radiology report and thus plays a crucial role in communication between radiologists and physicians. Research on radiology report summarization mostly focuses on generating the Impression section by summarizing information from the Findings section, which typically details the radiologist’s observations in the radiology images. Recent work start to explore how to incorporate radiology images as input to multimodal summarization models, with the assumption that it can improve generated summary quality, as it contains richer information. However, the real effectiveness of radiology images remains unclear. To answer this, we conduct a thorough analysis to understand whether current multimodal models can utilize radiology images in summarizing Findings section. Our analysis reveals that current multimodal models often fail to effectively utilize radiology images. For example, masking the image input leads to minimal or no performance drop. Expert annotation study shows that radiology images are unnecessary when they write the Impression section.
Anthology ID:
2025.findings-emnlp.1040
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2025
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
19116–19131
Language:
URL:
https://aclanthology.org/2025.findings-emnlp.1040/
DOI:
Bibkey:
Cite (ACL):
Mong Yuan Sim, Wei Emma Zhang, Xiang Dai, Biaoyan Fang, Sarbin Ranjitkar, Arjun Burlakoti, Jamie Taylor, and Haojie Zhuang. 2025. The More, The Better? A Critical Study of Multimodal Context in Radiology Report Summarization. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 19116–19131, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
The More, The Better? A Critical Study of Multimodal Context in Radiology Report Summarization (Sim et al., Findings 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.findings-emnlp.1040.pdf
Checklist:
 2025.findings-emnlp.1040.checklist.pdf