Peng Zhang
Author directoryOther people with similar names: Peng Zhang, Peng Zhang, Peng Zhang
Unverified author pages with similar names: Peng Zhang
2026
StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance
Guanming Liu | Yuqi Ren | Hansu Gu | Peng Zhang | Weihang Wang | Jiahao Liu | Ning Gu | Tun Lu
Findings of the Association for Computational Linguistics: EMNLP 2026
Guanming Liu | Yuqi Ren | Hansu Gu | Peng Zhang | Weihang Wang | Jiahao Liu | Ning Gu | Tun Lu
Findings of the Association for Computational Linguistics: EMNLP 2026
A central role of personal-agent memory is to turn stored information and prior interactions into future-oriented assistance. In daily use, useful cues come from what the agent observes and how the user interacts with the agent, and the agent must carry them forward from the current request to similar future tasks. Existing memory benchmarks usually test dialogue recall or task improvement in isolation, leaving the trajectory from streaming observations to later assistance largely untested. We introduce StreamMemBench, a streaming benchmark that constructs a two-step task sequence around each evidence anchor from EgoLife egocentric streams. The initial task tests evidence use, while the follow-up task tests whether feedback and interaction experience are reused. Four metrics diagnose evidence recall, initial evidence use, feedback incorporation, and follow-up reuse. Experiments with eight memory systems across two backbones show that current systems often fail to use observed evidence or turn feedback into reliable follow-up behavior, even when evidence is stored or feedback is incorporated locally.
Fast-Slow Thinking RM: Efficient Integration of Scalar and Generative Reward Models
Jiayun Wu | Peng Zhang | Yuanyuan Lu | Shan Qu | Ning Gu | Tun Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jiayun Wu | Peng Zhang | Yuanyuan Lu | Shan Qu | Ning Gu | Tun Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Reward models (RMs) are critical for aligning Large Language Models via Reinforcement Learning from Human Feedback (RLHF). While Generative Reward Models (GRMs) achieve superior accuracy through chain-of-thought (CoT) reasoning, they incur substantial computational costs. Conversely, Scalar Reward Models (SRMs) offer efficiency but suffer from limited performance and adaptability in complex scenarios. We introduce Fast-Slow Thinking Reward Models (F/S-RM), a hybrid RM architecture inspired by Dual Process Theory. It trains a single model to integrate two distinct reward paradigms: scalar-style first-token pairwise judgment (fast thinking) and CoT-based judgment (slow thinking), regulated by a dual-confidence activation mechanism that determines when to activate slow thinking. Under hybrid inference, F/S-RM achieves state-of-the-art accuracy with an average score of 84.3 across benchmarks, while reducing token consumption by 22.5%. Code and data are publicly available.
VIRA-Bench: Diagnosing Pedagogical Contingency Understanding in Online Tutoring Videos
Weihang Wang | Hansu Gu | Guanming Liu | Peng Zhang | Ning Gu | Tun Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Weihang Wang | Hansu Gu | Guanming Liu | Peng Zhang | Ning Gu | Tun Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Understanding online tutoring requires more than recognizing isolated events. A model must connect what a teacher says or does to how the learner responds and how the teacher subsequently adapts. Existing video benchmarks emphasize event and temporal understanding, while educational-dialogue benchmarks are largely transcript-based, leaving this interactional contingency underexplored. We introduce VIRA-Bench, a diagnostic benchmark for short-horizon pedagogical contingency in one-to-one tutoring videos. Grounded in the Initiation–Response–Follow-up structure and contingent teaching theory, VIRA-Bench contains 1,005 clips and 1,322 multiple-choice questions organized into three layers: Observable Evidence Grounding, Interaction Contingency Reasoning, and Support-Response Modeling. Items are built through researcher-led evidence auditing and expert verification to keep questions tied to observable interaction evidence. Across 11 multimodal large language models, the best model reaches 61.3% accuracy, compared with 91.0% on a stratified human subset, with interaction contingency reasoning emerging as the most consistent weakness. Controlled evidence interventions reveal distinct dependencies across layers. Removing video hurts observable grounding most, while disrupting interaction order hurts contingency reasoning most. Additional controls show that this distinction survives free-response evaluation and cannot be explained by question-option shortcuts or video–transcript misalignment. By separating visual grounding from order-sensitive interaction reasoning, VIRA-Bench exposes diagnostic differences that aggregate video-QA scores obscure.
2025
Value Compass Benchmarks: A Comprehensive, Generative and Self-Evolving Platform for LLMs’ Value Evaluation
Jing Yao | Xiaoyuan Yi | Shitong Duan | Jindong Wang | Yuzhuo Bai | Muhua Huang | Yang Ou | Scarlett Li | Peng Zhang | Tun Lu | Zhicheng Dou | Maosong Sun | James Evans | Xing Xie
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
Jing Yao | Xiaoyuan Yi | Shitong Duan | Jindong Wang | Yuzhuo Bai | Muhua Huang | Yang Ou | Scarlett Li | Peng Zhang | Tun Lu | Zhicheng Dou | Maosong Sun | James Evans | Xing Xie
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
As large language models (LLMs) are gradually integrated into human daily life, assessing their underlying values becomes essential for understanding their risks and alignment with specific preferences. Despite growing efforts, current value evaluation methods face two key challenges. C1. Evaluation Validity: Static benchmarks fail to reflect intended values or yield informative results due to data contamination or a ceiling effect. C2. Result Interpretation: They typically reduce the pluralistic and often incommensurable values to one-dimensional scores, which hinders users from gaining meaningful insights and guidance. To address these challenges, we present Value Compass Benchmarks, the first dynamic, online and interactive platform specially devised for comprehensive value diagnosis of LLMs. It (1) grounds evaluations in multiple basic value systems from social science; (2) develops a generative evolving evaluation paradigm that automatically creates real-world test items co-evolving with ever-advancing LLMs; (3) offers multi-faceted result interpretation, including (i) fine-grained scores and case studies across 27 value dimensions for 33 leading LLMs, (ii) customized comparisons, and (iii) visualized analysis of LLMs’ alignment with cultural values. We hope Value Compass Benchmarks serves as a navigator for further enhancing LLMs’ safety and alignment, benefiting their responsible and adaptive development.
2024
Negating Negatives: Alignment with Human Negative Samples via Distributional Dispreference Optimization
Shitong Duan | Xiaoyuan Yi | Peng Zhang | Yan Liu | Zheng Liu | Tun Lu | Xing Xie | Ning Gu
Findings of the Association for Computational Linguistics: EMNLP 2024
Shitong Duan | Xiaoyuan Yi | Peng Zhang | Yan Liu | Zheng Liu | Tun Lu | Xing Xie | Ning Gu
Findings of the Association for Computational Linguistics: EMNLP 2024
Large language models (LLMs) have revolutionized the role of AI, yet pose potential social risks. To steer LLMs towards human preference, alignment technologies have been introduced and gained increasing attention. Nevertheless, existing methods heavily rely on high-quality positive-negative training pairs, suffering from noisy positive responses that are barely distinguishable from negative ones. Given recent LLMs’ proficiency in generating helpful responses, this work pivots towards a new research question: can we achieve alignment using solely human-annotated negative samples, preserving helpfulness while reducing harmfulness? For this purpose, we propose Distributional Dispreference Optimization (D2O), which maximizes the discrepancy between dispreferred responses and the generated non-negative ones. In this way, D2O effectively eschews harmful information without incorporating noisy positive samples, while avoiding collapse using self-generated responses as anchors. We demonstrate that D2O can be regarded as learning a distributional preference model reflecting human dispreference against negative responses, which is theoretically an upper bound of the instance-level DPO. Extensive experiments manifest that our method achieves comparable generation quality and surpasses the latest strong baselines in producing less harmful and more informative responses with better training stability and faster convergence.