Hallucinations in large multimodal models (LMMs), where responses appear correct but are factually wrong, limit their reliability in practical applications. This paper focuses on hallucinations in video understanding, a more dynamic and challenging domain than static modalities. We present HAVEN, a comprehensive benchmark with 6K questions spanning three dimensions: hallucination causes, aspects, and question formats. Then, we systematically evaluate 18 LMMs and analyze seven influential factors affecting hallucinations, including video duration, model size, and reasoning capabilities. Additionally, we explore two reasoning-based strategies to reduce hallucinations in LMMs: (1) text-based reasoning via rejection fine-tuning on reasoning datasets and segment-weighted Direct Preference Optimization, and (2) video-based reasoning through a tool-augmented frame-level operations on video optimized via supervised tool learning and Hallucination-aware Group Relative Policy Optimization (HGRPO). Extensive experiments and analyses demonstrate the effectiveness. Remarkably, video-based reasoning strategy improves the Qwen2.5-VL-7B-Instruct by 10.16% in accuracy and reduces the bias score by 4.89%.

Vision-Language Models (VLMs) excel at tasks such as image captioning and visual question answering but frequently produce hallucinated outputs that deviate from the actual visual input or prompt. While prior work links hallucination to biases in data or representation, their causal origins remain unclear. We propose a causal framework to analyze and mitigate hallucination in VLMs. Our key hypothesis is that hallucinations arise from unintended direct influences of the vision or text modality that bypass the intended multi-modal fusion. To examine this, we construct a causal graph of the VLM and use counterfactual analysis to estimate the Natural Direct Effect (NDE) of each modality and their interaction. By systematically identifying and suppressing these direct effects, we encourage outputs that are more faithfully grounded in true cross-modal reasoning. Our approach consists of three steps: (1) designing structural causal graphs to distinguish correct fusion pathways from spurious modality shortcuts, (2) estimating modality-specific and cross-modal NDE using perturbed image representations, hallucinated text embeddings, and degraded visual inputs, and (3) implementing a test-time intervention module to dynamically adjust the model’s dependence on each modality. Experimental results demonstrate that our method significantly reduces hallucination while preserving task performance, providing a robust and interpretable framework for improving VLM reliability.