Jianjun Gao
Author directory2026
EgoErrorVQA: Assess Egocentric Comprehension Capabilities through Procedural Errors for Ego-Agentic AI
Junlong li | Junxi LI | Jianjun Gao | Chen Cai | Lap-Pui Chau | Yi Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Junlong li | Junxi LI | Jianjun Gao | Chen Cai | Lap-Pui Chau | Yi Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
The majority of our everyday activities are procedural and consist of sequences of interdependent steps. However, existing benchmarks for Visual Agents and Visual Language Models (VLMs) overlook the evaluation of their procedural comprehension ability from an egocentric visual perspective, particularly for detecting procedural errors, a critical capability for everyday assistance. To bridge this gap, the EgoErrorVQA task is firstly proposed for egocentric procedural comprehension with explicit procedural errors modeling. Besides, we develop a user-friendly evaluator agent based on the Agent2Agent (A2A) protocol, enabling rigorous and standardized evaluation of visual agents through VQA-based interaction. A range of models are evaluated using both open-ended and multiple-choice questions, revealing persistent weaknesses in handling procedural errors and error types. Moreover, we introduce Ego-ADR, an Adaptive Decoupled Reasoning framework that decouples complex procedural reasoning to enhance models’ understanding of procedural errors. It achieves consistent performance gains over the selected baselines and attains state-of-the-art results on several metrics under comparable settings. Code: https://github.com/z1oong/EgoErrorVQA
2024
Empowering Large Language Model for Continual Video Question Answering with Collaborative Prompting
Chen Cai | Zheng Wang | Jianjun Gao | Wenyang Liu | Ye Lu | Runzhong Zhang | Kim-Hui Yap
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Chen Cai | Zheng Wang | Jianjun Gao | Wenyang Liu | Ye Lu | Runzhong Zhang | Kim-Hui Yap
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
In recent years, the rapid increase in online video content has underscored the limitations of static Video Question Answering (VideoQA) models trained on fixed datasets, as they struggle to adapt to new questions or tasks posed by newly available content. In this paper, we explore the novel challenge of VideoQA within a continual learning framework, and empirically identify a critical issue: fine-tuning a large language model (LLM) for a sequence of tasks often results in catastrophic forgetting. To address this, we propose Collaborative Prompting (ColPro), which integrates specific question constraint prompting, knowledge acquisition prompting, and visual temporal awareness prompting. These prompts aim to capture textual question context, visual content, and video temporal dynamics in VideoQA, a perspective underexplored in prior research. Experimental results on the NExT-QA and DramaQA datasets show that ColPro achieves superior performance compared to existing approaches, achieving 55.14% accuracy on NExT-QA and 71.24% accuracy on DramaQA, highlighting its practical relevance and effectiveness.