KIA: Knowledge-Guided Implicit Vision-Language Alignment for Chest X-Ray Report Generation

Heng Yin, Shanlin Zhou, Pandong Wang, Zirui Wu, Yongtao Hao


Abstract
Report generation (RG) faces challenges in understanding complex medical images and establishing cross-modal semantic alignment in radiology image-report pairs. Previous methods often overlook fine-grained cross-modal interaction, leading to insufficient understanding of detailed information. Recently, various large multimodal models have been proposed for image-text tasks. However, such models still underperform on rare domain tasks like understanding complex medical images. To address these limitations, we develop a new framework of Knowledge-guided Implicit vision-language Alignment for radiology report generation, named KIA. To better understand medical reports and images and build alignment between them, multi-task implicit alignment is creatively introduced, forming comprehensive understanding of medical images and reports. Additionally, to further meet medical refinement requirements, we design novel masking strategies guided by medical knowledge to enhance pathological observation and anatomical landm
Anthology ID:
2025.coling-main.276
Volume:
Proceedings of the 31st International Conference on Computational Linguistics
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4096–4108
Language:
URL:
https://aclanthology.org/2025.coling-main.276/
DOI:
Bibkey:
Cite (ACL):
Heng Yin, Shanlin Zhou, Pandong Wang, Zirui Wu, and Yongtao Hao. 2025. KIA: Knowledge-Guided Implicit Vision-Language Alignment for Chest X-Ray Report Generation. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4096–4108, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
KIA: Knowledge-Guided Implicit Vision-Language Alignment for Chest X-Ray Report Generation (Yin et al., COLING 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.coling-main.276.pdf