A Skill-augmented Agentic Framework and Benchmark for Multi-Video Understanding

Yue Zhang, Liqiang Jing, Jia Li, Yapeng Tian, Xinya Du, Yunhui Guo, Vibhav Giridhar Gogate


Abstract
Multimodal Large Language Models have achieved strong performance in single-video understanding, yet their ability to reason across multiple videos remains limited. Existing approaches typically concatenate multiple videos into a single input and perform direct inference, which introduces training–inference mismatch, information loss from frame compression, and a lack of explicit cross-video coordination. Meanwhile, current multi-video benchmarks primarily emphasize event-level comparison, leaving identity-level matching, fine-grained discrimination, and structured multi-step reasoning underexplored. To address these gaps, we introduce MVX-Bench, a Multi-Video Cross-Dimension Benchmark that reformulates 11 classical computer vision tasks into a unified multi-video question-answering framework, comprising 1,442 questions over 4,255 videos from diverse real-world datasets. We further propose SAMA, a Skill-Augmented Agentic Framework for Multi-Video Understanding, which integrates visual tools, task-specific skills, and a conflict-aware verification mechanism to support iterative and structured reasoning. Experiments show that SAMA improves substantially over its visual perception backbone and achieves the best average accuracy on MVX-Bench. Ablations confirm the benefits of skill-guided orchestration and conflict-aware verification.
Anthology ID:
2026.findings-emnlp.429
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
9032–9057
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.429/
DOI:
Bibkey:
Cite (ACL):
Yue Zhang, Liqiang Jing, Jia Li, Yapeng Tian, Xinya Du, Yunhui Guo, and Vibhav Giridhar Gogate. 2026. A Skill-augmented Agentic Framework and Benchmark for Multi-Video Understanding. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 9032–9057, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
A Skill-augmented Agentic Framework and Benchmark for Multi-Video Understanding (Zhang et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.429.pdf