Medical vision-language models are typically trained by aligning whole images with entire radiology reports, assuming that global correspondence is sufficient for learning clinically meaningful representations. We argue that effective medical representation learning requires pathology-localized supervision, which explicitly aligns specific disease observations with their corresponding anatomical image regions. To enable this at scale, we propose a fully automated pipeline that decomposes radiology reports into atomic disease observations, identifies relevant regions, and pairs them with corresponding image regions without manual annotation, resulting in a fine-grained dataset, LocalizedCXR, with over 6 million region–caption pairs. Through extensive experiments, we demonstrate that our pathology-localized semantic supervision significantly improve performance in cross-modal retrieval, image classification, and phrase grounding of radiology findings. Our results suggest that moving beyond global alignment to disease-aware region-level alignment is essential for developing effective medical vision-language representation learning.

Vision-language models align frozen vision encoders to language models using only a next-token prediction loss, an asymmetric signal that discards rich visual signals not captured in the captions. We propose GenDA (Generative Distillation Alignment), a novel training framework that augments the captioning loss with a generative distillation objective from a frozen text-to-image diffusion model. GenDA concatenates selected visual tokens with the encoded caption to form a joint conditioning signal, encouraging these tokens to encode visual information not already captured by the caption. Across five vision encoders and four language models, GenDA gains 3-9 CIDEr points over LLaVA-1.5 on five captioning benchmarks. These gains also transfer to downstream VLM benchmarks while preserving the encoders’ discriminative performance. Visual preservation and modality-gap analyses suggest that visual information is retained deeper within the LLM, while visual features become more closely aligned with the language representation space.

Large Vision-Language Models (LVLMs) achieve strong multimodal reasoning capabilities, yet are still prone to object hallucination - often generating content that is not grounded by the visual input. Existing attention-based training-free mitigating methods rely on visual attention heads identified by high attention weights or sensitivity to visual token perturbations. However, we show that these identified visual attention heads are not necessarily responsible for visual grounding, and using this as guidance is suboptimal. Motivated by this observation, we propose a novel training-free decoding method that identifies visually sensitive heads by counterfactual region masking. We find that these heads provide a clear signal when visual grounding is required and can therefore be used to mitigate hallucination by guiding the suppression of hallucination heads. Across multiple LVLMs and benchmark image captioning and VQA datasets, our method achieves best or near-best performance in reducing hallucination while preserving generation quality.

The Impression section of a radiology report summarizes critical findings of a radiology report and thus plays a crucial role in communication between radiologists and physicians. Research on radiology report summarization mostly focuses on generating the Impression section by summarizing information from the Findings section, which typically details the radiologist’s observations in the radiology images. Recent work start to explore how to incorporate radiology images as input to multimodal summarization models, with the assumption that it can improve generated summary quality, as it contains richer information. However, the real effectiveness of radiology images remains unclear. To answer this, we conduct a thorough analysis to understand whether current multimodal models can utilize radiology images in summarizing Findings section. Our analysis reveals that current multimodal models often fail to effectively utilize radiology images. For example, masking the image input leads to minimal or no performance drop. Expert annotation study shows that radiology images are unnecessary when they write the Impression section.

Vision-language models (VLMs) integrate textual and visual information, enabling the model to process visual inputs and leverage visual information to generate predictions. Such models are demanding for tasks such as visual question answering, image captioning, and visual grounding. However, some recent work found that VLMs often rely heavily on textual information, ignoring visual information, but are still able to achieve competitive performance in vision-language (VL) tasks. This survey reviews modality collapse analysis work to provide insights into the reason for this unintended behavior. It also reviews probing studies for fine-grained vision-language understanding, presenting current findings on information encoded in VL representations and highlighting potential directions for future research.

Lay summarisation aims at generating a summary for non-expert audience which allows them to keep updated with latest research in a specific field. Despite the significant advancements made in the field of text summarisation, lay summarisation remains relatively under-explored. We present a comprehensive set of experiments and analysis to investigate the effectiveness of existing pre-trained language models in generating lay summaries. When evaluate our models using a BioNLP Shared Task, BioLaySumm, our submission ranked second for the relevance criteria and third overall among 21 competing teams.

Multi-document summarization (MDS) is a process of generating an informative and concise summary from multiple topic-related documents. Many studies have analyzed the quality of MDS dataset or models, however no work has been done from the perspective of topic preservation. In this work, we fill the gap by performing an empirical analysis on two MDS datasets and study topic preservation on generated summaries from 8 MDS models. Our key findings include i) Multi-News dataset has better gold summaries compared to Multi-XScience in terms of its topic distribution consistency and ii) Extractive approaches perform better than abstractive approaches in preserving topic information from source documents. We hope our findings could help develop a summarization model that can generate topic-focused summary and also give inspiration to researchers in creating dataset for such challenging task.