Ning Gu
Author directory2026
StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance
Guanming Liu | Yuqi Ren | Hansu Gu | Peng Zhang | Weihang Wang | Jiahao Liu | Ning Gu | Tun Lu
Findings of the Association for Computational Linguistics: EMNLP 2026
Guanming Liu | Yuqi Ren | Hansu Gu | Peng Zhang | Weihang Wang | Jiahao Liu | Ning Gu | Tun Lu
Findings of the Association for Computational Linguistics: EMNLP 2026
A central role of personal-agent memory is to turn stored information and prior interactions into future-oriented assistance. In daily use, useful cues come from what the agent observes and how the user interacts with the agent, and the agent must carry them forward from the current request to similar future tasks. Existing memory benchmarks usually test dialogue recall or task improvement in isolation, leaving the trajectory from streaming observations to later assistance largely untested. We introduce StreamMemBench, a streaming benchmark that constructs a two-step task sequence around each evidence anchor from EgoLife egocentric streams. The initial task tests evidence use, while the follow-up task tests whether feedback and interaction experience are reused. Four metrics diagnose evidence recall, initial evidence use, feedback incorporation, and follow-up reuse. Experiments with eight memory systems across two backbones show that current systems often fail to use observed evidence or turn feedback into reliable follow-up behavior, even when evidence is stored or feedback is incorporated locally.
Fast-Slow Thinking RM: Efficient Integration of Scalar and Generative Reward Models
Jiayun Wu | Peng Zhang | Yuanyuan Lu | Shan Qu | Ning Gu | Tun Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jiayun Wu | Peng Zhang | Yuanyuan Lu | Shan Qu | Ning Gu | Tun Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Reward models (RMs) are critical for aligning Large Language Models via Reinforcement Learning from Human Feedback (RLHF). While Generative Reward Models (GRMs) achieve superior accuracy through chain-of-thought (CoT) reasoning, they incur substantial computational costs. Conversely, Scalar Reward Models (SRMs) offer efficiency but suffer from limited performance and adaptability in complex scenarios. We introduce Fast-Slow Thinking Reward Models (F/S-RM), a hybrid RM architecture inspired by Dual Process Theory. It trains a single model to integrate two distinct reward paradigms: scalar-style first-token pairwise judgment (fast thinking) and CoT-based judgment (slow thinking), regulated by a dual-confidence activation mechanism that determines when to activate slow thinking. Under hybrid inference, F/S-RM achieves state-of-the-art accuracy with an average score of 84.3 across benchmarks, while reducing token consumption by 22.5%. Code and data are publicly available.
VIRA-Bench: Diagnosing Pedagogical Contingency Understanding in Online Tutoring Videos
Weihang Wang | Hansu Gu | Guanming Liu | Peng Zhang | Ning Gu | Tun Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Weihang Wang | Hansu Gu | Guanming Liu | Peng Zhang | Ning Gu | Tun Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Understanding online tutoring requires more than recognizing isolated events. A model must connect what a teacher says or does to how the learner responds and how the teacher subsequently adapts. Existing video benchmarks emphasize event and temporal understanding, while educational-dialogue benchmarks are largely transcript-based, leaving this interactional contingency underexplored. We introduce VIRA-Bench, a diagnostic benchmark for short-horizon pedagogical contingency in one-to-one tutoring videos. Grounded in the Initiation–Response–Follow-up structure and contingent teaching theory, VIRA-Bench contains 1,005 clips and 1,322 multiple-choice questions organized into three layers: Observable Evidence Grounding, Interaction Contingency Reasoning, and Support-Response Modeling. Items are built through researcher-led evidence auditing and expert verification to keep questions tied to observable interaction evidence. Across 11 multimodal large language models, the best model reaches 61.3% accuracy, compared with 91.0% on a stratified human subset, with interaction contingency reasoning emerging as the most consistent weakness. Controlled evidence interventions reveal distinct dependencies across layers. Removing video hurts observable grounding most, while disrupting interaction order hurts contingency reasoning most. Additional controls show that this distinction survives free-response evaluation and cannot be explained by question-option shortcuts or video–transcript misalignment. By separating visual grounding from order-sensitive interaction reasoning, VIRA-Bench exposes diagnostic differences that aggregate video-QA scores obscure.
2024
Negating Negatives: Alignment with Human Negative Samples via Distributional Dispreference Optimization
Shitong Duan | Xiaoyuan Yi | Peng Zhang | Yan Liu | Zheng Liu | Tun Lu | Xing Xie | Ning Gu
Findings of the Association for Computational Linguistics: EMNLP 2024
Shitong Duan | Xiaoyuan Yi | Peng Zhang | Yan Liu | Zheng Liu | Tun Lu | Xing Xie | Ning Gu
Findings of the Association for Computational Linguistics: EMNLP 2024
Large language models (LLMs) have revolutionized the role of AI, yet pose potential social risks. To steer LLMs towards human preference, alignment technologies have been introduced and gained increasing attention. Nevertheless, existing methods heavily rely on high-quality positive-negative training pairs, suffering from noisy positive responses that are barely distinguishable from negative ones. Given recent LLMs’ proficiency in generating helpful responses, this work pivots towards a new research question: can we achieve alignment using solely human-annotated negative samples, preserving helpfulness while reducing harmfulness? For this purpose, we propose Distributional Dispreference Optimization (D2O), which maximizes the discrepancy between dispreferred responses and the generated non-negative ones. In this way, D2O effectively eschews harmful information without incorporating noisy positive samples, while avoiding collapse using self-generated responses as anchors. We demonstrate that D2O can be regarded as learning a distributional preference model reflecting human dispreference against negative responses, which is theoretically an upper bound of the instance-level DPO. Extensive experiments manifest that our method achieves comparable generation quality and surpasses the latest strong baselines in producing less harmful and more informative responses with better training stability and faster convergence.
2023
An Intent-based and Annotation-free Method for Duplicate Question Detection in CQA Forums
Yubo Shu | Hansu Gu | Peng Zhang | Tun Lu | Ning Gu
Findings of the Association for Computational Linguistics: EMNLP 2023
Yubo Shu | Hansu Gu | Peng Zhang | Tun Lu | Ning Gu
Findings of the Association for Computational Linguistics: EMNLP 2023
With the advent of large language models (LLMs), Community Question Answering (CQA) forums offer well-curated questions and answers that can be utilized for instruction-tuning, effectively training LLMs to be aligned with human intents. However, the issue of duplicate questions arises as the volume of content within CQA continues to grow, posing a threat to content quality. Recent research highlights the benefits of detecting and eliminating duplicate content. It not only enhances the LLMs’ ability to generalize across diverse intents but also improves the efficiency of training data utilization while addressing concerns related to information leakage. However, existing methods for detecting duplicate questions in CQA typically rely on generic text-pair matching models, overlooking the intent behind the questions. In this paper, we propose a novel intent-based duplication detector named Intent-DQD that comprehensively leverages intent information to address the problem of duplicate question detection in CQA. Intent-DQD first leverages the characteristics in CQA forums and extracts training labels to recognize and match intents without human annotation. Intent-DQD then effectively aggregates intent-level relations and establishes question-level relations to enable intent-aware duplication detection. Experimental results on fifteen distinct domains from both CQADupStack and Stack Overflow datasets demonstrate the effectiveness of Intent-DQD. Reproducible codes and datasets will be released upon publication of the paper.