Mohammed Bennamoun
Author directory2026
Self-Supervised On-Policy Distillation for Omni-Modal Video Captioning
Zinuo Li | Yongxin Guo | Xin Li | Hao Tan | Zhao Zhong | Liefeng Bo | Mohammed Bennamoun | Farid Boussaid | Qiuhong Ke
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zinuo Li | Yongxin Guo | Xin Li | Hao Tan | Zhao Zhong | Liefeng Bo | Mohammed Bennamoun | Farid Boussaid | Qiuhong Ke
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Video captioning directly reflects a Video Large Language Model’s perception, making caption fidelity a strong proxy for audio-visual grounding. Existing approaches often rely on costly annotations or stronger external teachers, limiting scalability. In this paper, we instead explore whether a VideoLLM can improve its own captions by exploiting structure already present within the model through self-supervision. Our exploration builds on two asymmetries we observe in current VideoLLMs. The first is a task asymmetry: captioning requires broad open-ended perception, whereas question answering narrows reasoning to specific dimensions. The second is an information asymmetry: the same model reasons more reliably with richer context or higher-fidelity audio-visual inputs. Motivated by these observations, we propose SODA (Self-Supervised On-Policy Distillation via Dual-Stream Auditing), a framework that enables a VideoLLM to critique and refine its own captions. A student first produces an initial caption from a low-budget video view, followed by a dual-stream audit: a Caption QA Stream identifies factual errors, and a Video QA Stream detects omitted content. Conditioned on this Revision QA Bank, a teacher using a larger input budget rewrites the caption, and the refined outputs are distilled back into the student. Experiments suggest that, despite training only on the captioning objective, SODA improves caption fidelity, reduces hallucination, and remains competitive on general video understanding.
2018
NovelPerspective: Identifying Point of View Characters
Lyndon White | Roberto Togneri | Wei Liu | Mohammed Bennamoun
Proceedings of ACL 2018, System Demonstrations
Lyndon White | Roberto Togneri | Wei Liu | Mohammed Bennamoun
Proceedings of ACL 2018, System Demonstrations
We present NovelPerspective: a tool to allow consumers to subset their digital literature, based on point of view (POV) character. Many novels have multiple main characters each with their own storyline running in parallel. A well-known example is George R. R. Martin’s novel: “A Game of Thrones”, and others from that series. Our tool detects the main character that each section is from the POV of, and allows the user to generate a new ebook with only those sections. This gives consumers new options in how they consume their media; allowing them to pursue the storylines sequentially, or skip chapters about characters they find boring. We present two heuristic-based baselines, and two machine learning based methods for the detection of the main character.
Learning-based Composite Metrics for Improved Caption Evaluation
Naeha Sharif | Lyndon White | Mohammed Bennamoun | Syed Afaq Ali Shah
Proceedings of ACL 2018, Student Research Workshop
Naeha Sharif | Lyndon White | Mohammed Bennamoun | Syed Afaq Ali Shah
Proceedings of ACL 2018, Student Research Workshop
The evaluation of image caption quality is a challenging task, which requires the assessment of two main aspects in a caption: adequacy and fluency. These quality aspects can be judged using a combination of several linguistic features. However, most of the current image captioning metrics focus only on specific linguistic facets, such as the lexical or semantic, and fail to meet a satisfactory level of correlation with human judgements at the sentence-level. We propose a learning-based framework to incorporate the scores of a set of lexical and semantic metrics as features, to capture the adequacy and fluency of captions at different linguistic levels. Our experimental results demonstrate that composite metrics draw upon the strengths of stand-alone measures to yield improved correlation and accuracy.