Farid Boussaid
Author directory2026
Self-Supervised On-Policy Distillation for Omni-Modal Video Captioning
Zinuo Li | Yongxin Guo | Xin Li | Hao Tan | Zhao Zhong | Liefeng Bo | Mohammed Bennamoun | Farid Boussaid | Qiuhong Ke
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zinuo Li | Yongxin Guo | Xin Li | Hao Tan | Zhao Zhong | Liefeng Bo | Mohammed Bennamoun | Farid Boussaid | Qiuhong Ke
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Video captioning directly reflects a Video Large Language Model’s perception, making caption fidelity a strong proxy for audio-visual grounding. Existing approaches often rely on costly annotations or stronger external teachers, limiting scalability. In this paper, we instead explore whether a VideoLLM can improve its own captions by exploiting structure already present within the model through self-supervision. Our exploration builds on two asymmetries we observe in current VideoLLMs. The first is a task asymmetry: captioning requires broad open-ended perception, whereas question answering narrows reasoning to specific dimensions. The second is an information asymmetry: the same model reasons more reliably with richer context or higher-fidelity audio-visual inputs. Motivated by these observations, we propose SODA (Self-Supervised On-Policy Distillation via Dual-Stream Auditing), a framework that enables a VideoLLM to critique and refine its own captions. A student first produces an initial caption from a low-budget video view, followed by a dual-stream audit: a Caption QA Stream identifies factual errors, and a Video QA Stream detects omitted content. Conditioned on this Revision QA Bank, a teacher using a larger input budget rewrites the caption, and the refined outputs are distilled back into the student. Experiments suggest that, despite training only on the captioning objective, SODA improves caption fidelity, reduces hallucination, and remains competitive on general video understanding.