LOTUS: Language-Vision Optimal Transport for Unified Report Synthesis

Xuhui Li, Jiachen Liu


Abstract
Automated radiology report generation can reduce reporting burden, but existing encoder-decoder systems still rely on soft cross-attention that offers no geometric guarantee that generated tokens consume the correct visual evidence. We present LOTUS, an optimal-transport framework that places alignment supervision directly at the decoder evidence interface. LOTUS introduces Cross-Attention OT (CA-OT) to align decoder token states with visual patches, and extends it with Prototype OT (Proto-OT) for structure-level alignment between visual and textual codebooks. Experiments on IU X-Ray and ROCO, complemented by a scalability study on MIMIC-CXR-RRG, show that decoder-side OT improves macro-level clinical-label agreement while maintaining competitive language quality. In multi-seed IU X-Ray evaluation, LOTUS-Deep improves macro clinical F1 from 0.162 to 0.201 over the CE baseline, and on ROCO it yields micro-F1 gains on a thoracic-label proxy (0.395 to 0.600). We also provide a population-level risk-gap bound that justifies expected OT regularization as a principled cross-modal training signal.
Anthology ID:
2026.findings-aacl.86
Volume:
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1746–1762
Language:
URL:
https://aclanthology.org/2026.findings-aacl.86/
DOI:
Bibkey:
Cite (ACL):
Xuhui Li and Jiachen Liu. 2026. LOTUS: Language-Vision Optimal Transport for Unified Report Synthesis. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026, pages 1746–1762, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
LOTUS: Language-Vision Optimal Transport for Unified Report Synthesis (Li & Liu, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-aacl.86.pdf