2026

Hallucinations in large multimodal models (LMMs), where responses appear correct but are factually wrong, limit their reliability in practical applications. This paper focuses on hallucinations in video understanding, a more dynamic and challenging domain than static modalities. We present HAVEN, a comprehensive benchmark with 6K questions spanning three dimensions: hallucination causes, aspects, and question formats. Then, we systematically evaluate 18 LMMs and analyze seven influential factors affecting hallucinations, including video duration, model size, and reasoning capabilities. Additionally, we explore two reasoning-based strategies to reduce hallucinations in LMMs: (1) text-based reasoning via rejection fine-tuning on reasoning datasets and segment-weighted Direct Preference Optimization, and (2) video-based reasoning through a tool-augmented frame-level operations on video optimized via supervised tool learning and Hallucination-aware Group Relative Policy Optimization (HGRPO). Extensive experiments and analyses demonstrate the effectiveness. Remarkably, video-based reasoning strategy improves the Qwen2.5-VL-7B-Instruct by 10.16% in accuracy and reduces the bias score by 4.89%.

2025

Change captioning aims to describe differences between a pair of images using natural language. However, learning effective difference representations is highly challenging due to distractors such as illumination and viewpoint changes. To address this, we propose a change-entity-guided disentanglement network that explicitly learns difference representations while mitigating the impact of distractors. Specifically, we first design a change entity retrieval module to identify key objects involved in the change from a textual perspective. Then, we introduce a difference representation enhancement module that strengthens the learned features, disentangling genuine differences from background variations. To further refine the generation process, we incorporate a gated Transformer decoder, which dynamically integrates both visual difference and textual change-entity information. Extensive experiments on CLEVR-Change, CLEVR-DC and Spot-the-Diff datasets demonstrate that our method outperforms existing approaches, achieving state-of-the-art performance. The code is available at https://github.com/yili-19/CHEER

2024

Multi-change captioning aims to describe complex and coupled changes within an image pair in natural language. Compared with single-change captioning, this task requires the model to have higher-level cognition ability to reason an arbitrary number of changes. In this paper, we propose a novel context-aware difference distilling (CARD) network to capture all genuine changes for yielding sentences. Given an image pair, CARD first decouples context features that aggregate all similar/dissimilar semantics, termed common/difference context features. Then, the consistency and independence constraints are designed to guarantee the alignment/discrepancy of common/difference context features. Further, the common context features guide the model to mine locally unchanged features, which are subtracted from the pair to distill locally difference features. Next, the difference context features augment the locally difference features to ensure that all changes are distilled. In this way, we obtain an omni-representation of all changes, which is translated into linguistic sentences by a transformer decoder. Extensive experiments on three public datasets show CARD performs favourably against state-of-the-art methods. The code is available at https://github.com/tuyunbin/CARD.

2016