Guanming Liu
Author directory2026
StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance
Guanming Liu | Yuqi Ren | Hansu Gu | Peng Zhang | Weihang Wang | Jiahao Liu | Ning Gu | Tun Lu
Findings of the Association for Computational Linguistics: EMNLP 2026
Guanming Liu | Yuqi Ren | Hansu Gu | Peng Zhang | Weihang Wang | Jiahao Liu | Ning Gu | Tun Lu
Findings of the Association for Computational Linguistics: EMNLP 2026
A central role of personal-agent memory is to turn stored information and prior interactions into future-oriented assistance. In daily use, useful cues come from what the agent observes and how the user interacts with the agent, and the agent must carry them forward from the current request to similar future tasks. Existing memory benchmarks usually test dialogue recall or task improvement in isolation, leaving the trajectory from streaming observations to later assistance largely untested. We introduce StreamMemBench, a streaming benchmark that constructs a two-step task sequence around each evidence anchor from EgoLife egocentric streams. The initial task tests evidence use, while the follow-up task tests whether feedback and interaction experience are reused. Four metrics diagnose evidence recall, initial evidence use, feedback incorporation, and follow-up reuse. Experiments with eight memory systems across two backbones show that current systems often fail to use observed evidence or turn feedback into reliable follow-up behavior, even when evidence is stored or feedback is incorporated locally.
VIRA-Bench: Diagnosing Pedagogical Contingency Understanding in Online Tutoring Videos
Weihang Wang | Hansu Gu | Guanming Liu | Peng Zhang | Ning Gu | Tun Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Weihang Wang | Hansu Gu | Guanming Liu | Peng Zhang | Ning Gu | Tun Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Understanding online tutoring requires more than recognizing isolated events. A model must connect what a teacher says or does to how the learner responds and how the teacher subsequently adapts. Existing video benchmarks emphasize event and temporal understanding, while educational-dialogue benchmarks are largely transcript-based, leaving this interactional contingency underexplored. We introduce VIRA-Bench, a diagnostic benchmark for short-horizon pedagogical contingency in one-to-one tutoring videos. Grounded in the Initiation–Response–Follow-up structure and contingent teaching theory, VIRA-Bench contains 1,005 clips and 1,322 multiple-choice questions organized into three layers: Observable Evidence Grounding, Interaction Contingency Reasoning, and Support-Response Modeling. Items are built through researcher-led evidence auditing and expert verification to keep questions tied to observable interaction evidence. Across 11 multimodal large language models, the best model reaches 61.3% accuracy, compared with 91.0% on a stratified human subset, with interaction contingency reasoning emerging as the most consistent weakness. Controlled evidence interventions reveal distinct dependencies across layers. Removing video hurts observable grounding most, while disrupting interaction order hurts contingency reasoning most. Additional controls show that this distinction survives free-response evaluation and cannot be explained by question-option shortcuts or video–transcript misalignment. By separating visual grounding from order-sensitive interaction reasoning, VIRA-Bench exposes diagnostic differences that aggregate video-QA scores obscure.