Junjun He
Author directory2026
Towards Anticipatory Agents for Streaming Video Understanding
Haolin Yang | Feilong Tang | Lingxiao Zhao | Xinlin Zhuang | Yifan Lu | Xiang An | Ming Hu | Xiaofeng Zhang | Abdalla Swikir | Junjun He | Zongyuan Ge | Muhammad Haris Khan | Imran Razzak
Findings of the Association for Computational Linguistics: EMNLP 2026
Haolin Yang | Feilong Tang | Lingxiao Zhao | Xinlin Zhuang | Yifan Lu | Xiang An | Ming Hu | Xiaofeng Zhang | Abdalla Swikir | Junjun He | Zongyuan Ge | Muhammad Haris Khan | Imran Razzak
Findings of the Association for Computational Linguistics: EMNLP 2026
Real-time streaming video understanding in domains such as autonomous driving and intelligent surveillance poses challenges beyond conventional offline video processing, requiring continuous perception, proactive decision making, and responsive interaction based on dynamically evolving visual content. However, existing methods rely on alternating perception-reaction or asynchronous triggers, lacking task-driven planning and future anticipation, which limits their real-time responsiveness and proactive decision making in evolving video streams. To this end, we propose a StreamAgent that anticipates the temporal intervals and spatial regions expected to contain future task-relevant information to enable proactive and goal-driven responses. Specifically, we integrate question semantics and historical observations through prompting the anticipatory agent to anticipate the temporal progression of key events, align current observations with the expected future evidence, and subsequently adjust the perception action (e.g., attending to task-relevant regions or continuously tracking in subsequent frames). To enable efficient inference, we design a streaming KV-cache memory mechanism that constructs a hierarchical memory structure for selective recall of relevant tokens, enabling efficient semantic retrieval while reducing the overhead of storing all tokens in the traditional KV-cache. Extensive experiments on streaming and long video understanding tasks demonstrate that our method outperforms existing methods in response accuracy and real-time efficiency, highlighting its practical value for real-world streaming scenarios.
THE-Tree: Can Tracing Historical Evolution Enhance Scientific Verification and Reasoning?
Xin Wang | Jiyao Liu | Yulong Xiao | Junzhi Ning | Hairong Zhang | Lihao Liu | Junjun He | Botian Shi | Kaicheng Yu
Findings of the Association for Computational Linguistics: EMNLP 2026
Xin Wang | Jiyao Liu | Yulong Xiao | Junzhi Ning | Hairong Zhang | Lihao Liu | Junjun He | Botian Shi | Kaicheng Yu
Findings of the Association for Computational Linguistics: EMNLP 2026
Large Language Models (LLMs) are accelerating scientific idea generation, but rigorously evaluating these numerous, often superficial, AI-generated propositions for novelty and factual accuracy is a critical bottleneck; manual verification is too slow. Existing validation methods are inadequate: LLMs as standalone verifiers may hallucinate and lack domain knowledge (our findings show ~60% unawareness of relevant papers in specific domains), while traditional citation networks lack explicit causality and narrative surveys are unstructured. This underscores a core challenge: the absence of structured, verifiable, and causally-linked historical data of scientific evolution. To address this, we introduce THE-Tree (Technology History Evolution Tree), a computational framework that constructs such domain-specific evolution trees from scientific literature. THE-Tree employs a search algorithm to explore evolutionary paths. During its node expansion, it utilizes a novel "Think-Verbalize-Cite-Verify" process: an LLM proposes potential advancements and cites supporting literature. Critically, each proposed evolutionary link is then validated for logical coherence and evidential support by a recovered natural language inference mechanism that interrogates the cited literature, ensuring that each step is grounded. We construct and validate 88 THE-Trees across diverse domains and release a benchmark dataset including up to 71k fact verifications covering 27k papers to foster further research. Experiments demonstrate that i) in graph completion, our THE-Tree improves hit@1 by 8% to 14% across multiple models compared to traditional citation networks; ii) for predicting future scientific developments, it improves hit@1 metric by nearly 10%; and iii) when combined with other methods, it boosts the performance of evaluating important scientific papers by almost 100%. By constructing explicit, verifiable pathways of scientific progression, THE-Tree provides a robust historical foundation for evaluating new hypotheses (human or AI-generated) and enables a computable science history, fostering evidence-based AI-driven scientific discovery.
Can MLLMs Perceive and Describe Medical Image Quality Beyond Scores?
Jiyao Liu | Jinjie Wei | Wanying Qu | Chenglong Ma | Junzhi Ning | Yunheng Li | Ying Chen | Jiafan Zhang | Pengcheng Chen | Xin Gao | Ming Hu | Xin Wang | Shujian Gao | Dingkang Yang | Zhongying Deng | Jin Ye | Lihao Liu | Junjun He | Ningsheng Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jiyao Liu | Jinjie Wei | Wanying Qu | Chenglong Ma | Junzhi Ning | Yunheng Li | Ying Chen | Jiafan Zhang | Pengcheng Chen | Xin Gao | Ming Hu | Xin Wang | Shujian Gao | Dingkang Yang | Zhongying Deng | Jin Ye | Lihao Liu | Junjun He | Ningsheng Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Medical Image Quality Assessment (IQA) serves as a critical safety gate for clinical AI, yet existing approaches rely on scalar metrics that fail to capture the descriptive, expert-like evaluation process central to clinical practice. We introduce MedQ-Bench, a comprehensive benchmark for language-based medical IQA evaluation with MLLMs, establishing a perception–description paradigm across 5 imaging modalities and over 40 quality attributes. The benchmark comprises two complementary tasks: MedQ-Perception probes low-level visual attribute discrimination via 2,600 expert-annotated perceptual queries; MedQ-Description covers both no-reference and comparison description sub-tasks with 708 expert-annotated samples, evaluated through a four-dimensional judging protocol. Comprehensive evaluation of 23 state-of-the-art MLLMs reveals that current models exhibit only preliminary and unstable capabilities, and highlights the urgent need for targeted optimization of MLLMs in medical IQA. Project Dataset: https://github.com/liujiyaoFDU/MedQ-Bench
2025
MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation
Haochen Xue | Feilong Tang | Ming Hu | Yexin Liu | Qidong Huang | Yulong Li | Chengzhi Liu | Zhongxing Xu | Chong Zhang | Chun-Mei Feng | Yutong Xie | Imran Razzak | Zongyuan Ge | Jionglong Su | Junjun He | Yu Qiao
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Haochen Xue | Feilong Tang | Ming Hu | Yexin Liu | Qidong Huang | Yulong Li | Chengzhi Liu | Zhongxing Xu | Chong Zhang | Chun-Mei Feng | Yutong Xie | Imran Razzak | Zongyuan Ge | Jionglong Su | Junjun He | Yu Qiao
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Recent multimodal large language models (MLLMs) have demonstrated significant potential in open-ended conversation, generating more accurate and personalized responses. However, their abilities to memorize, recall, and reason in sustained interactions within real-world scenarios remain underexplored. This paper introduces MMRC, a Multi-Modal Real-world Conversation benchmark for evaluating six core open-ended abilities of MLLMs: information extraction, multi-turn reasoning, information update, image management, memory recall, and answer refusal. With data collected from real-world scenarios, MMRC comprises 5,120 conversations and 28,720 corresponding manually labeled questions, posing a significant challenge to existing MLLMs. Evaluations on 20 MLLMs in MMRC indicate an accuracy drop during open-ended interactions. We identify four common failure patterns: long-term memory degradation, inadequacies in updating factual knowledge, accumulated assumption of error propagation, and reluctance to “say no.” To mitigate these issues, we propose a simple yet effective NOTE-TAKING strategy, which can record key information from the conversation and remind the model during its responses, enhancing conversational capabilities. Experiments across six MLLMs demonstrate significant performance improvements.
Search
Fix author
Co-authors
- Ming Hu 3
- Zongyuan Ge 2
- Jiyao Liu 2
- Lihao Liu 2
- Junzhi Ning 2
- Imran Razzak 2
- Feilong Tang 2
- Xin Wang 2
- Xiang An 1
- Pengcheng Chen 1
- Ying Chen 1
- Zhongying Deng 1
- Chun-Mei Feng 1
- Shujian Gao 1
- Xin Gao 1
- Qidong Huang 1
- Muhammad Haris Khan 1
- Yulong Li 1
- Yunheng Li 1
- Chengzhi Liu 1
- Yexin Liu 1
- Yifan Lu 1
- Chenglong Ma 1
- Yu Qiao 1
- Wanying Qu 1
- Botian Shi 1
- Jionglong Su 1
- Abdalla Swikir 1
- Jinjie Wei 1
- Yulong Xiao 1
- Yutong Xie 1
- Ningsheng Xu 1
- Zhongxing Xu 1
- Haochen Xue 1
- Dingkang Yang 1
- Haolin Yang 1
- Jin Ye 1
- Kaicheng Yu 1
- Chong Zhang 1
- Hairong Zhang 1
- Jiafan Zhang 1
- Xiaofeng Zhang 1
- Lingxiao Zhao 1
- Xinlin Zhuang 1