Marcella Cornia
Author directory2026
CounterVid: Counterfactual Video Generation for Mitigating Action and Temporal Hallucinations in Video-Language Models
Tobia Poppi | Burak Uzkent | Amanmeet Garg | Lucas Resque Porto | Garin N. Kessler | YZ Yang | Marcella Cornia | Lorenzo Baraldi | Rita Cucchiara | Florian Schiffers
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Tobia Poppi | Burak Uzkent | Amanmeet Garg | Lucas Resque Porto | Garin N. Kessler | YZ Yang | Marcella Cornia | Lorenzo Baraldi | Rita Cucchiara | Florian Schiffers
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Video-language models (VLMs) achieve strong multimodal understanding but remain prone to hallucinations, especially when reasoning about actions and temporal order. Existing mitigation strategies, such as textual filtering or random video perturbations, often fail to address the root cause: over-reliance on language priors rather than fine-grained visual dynamics. We propose a scalable framework for counterfactual video generation that synthesizes videos differing only in actions or temporal structure while preserving scene context. Our pipeline combines multimodal LLMs for action proposal and editing guidance with diffusion-based image and video models to generate semantic hard negatives at scale. Using this framework, we build CounterVid, a synthetic dataset of ~26k preference pairs constructed from short counterfactual action clips and targeting both action recognition and controlled action-sequence ordering. We further introduce MixDPO, a unified Direct Preference Optimization approach that jointly leverages textual and visual preferences. Across Qwen2.5-VL and InternVL3 backbones, MixDPO substantially improves action recognition and temporal ordering, yields gains on most standard video hallucination benchmarks, and largely preserves general video understanding. Our source code, trained models, and dataset are available at https://aimagelab.github.io/CounterVid.
2024
The Revolution of Multimodal Large Language Models: A Survey
Davide Caffagni | Federico Cocchi | Luca Barsellotti | Nicholas Moratelli | Sara Sarto | Lorenzo Baraldi | Lorenzo Baraldi | Marcella Cornia | Rita Cucchiara
Findings of the Association for Computational Linguistics: ACL 2024
Davide Caffagni | Federico Cocchi | Luca Barsellotti | Nicholas Moratelli | Sara Sarto | Lorenzo Baraldi | Lorenzo Baraldi | Marcella Cornia | Rita Cucchiara
Findings of the Association for Computational Linguistics: ACL 2024
Connecting text and visual modalities plays an essential role in generative intelligence. For this reason, inspired by the success of large language models, significant research efforts are being devoted to the development of Multimodal Large Language Models (MLLMs). These models can seamlessly integrate visual and textual modalities, while providing a dialogue-based interface and instruction-following capabilities. In this paper, we provide a comprehensive review of recent visual-based MLLMs, analyzing their architectural choices, multimodal alignment strategies, and training techniques. We also conduct a detailed analysis of these models across a wide range of tasks, including visual grounding, image generation and editing, visual understanding, and domain-specific applications. Additionally, we compile and describe training datasets and evaluation benchmarks, conducting comparisons among existing models in terms of performance and computational requirements. Overall, this survey offers a comprehensive overview of the current state of the art, laying the groundwork for future MLLMs.