Wei He
Author directoryOther people with similar names: Wei He, Wei He, Wei He
Unverified author pages with similar names: Wei He
2026
SAGE: An Extensible Benchmark for Service Agent Graph-guided Evaluation
Ling Shi | Yuqin Dai | Ziyin Wang | Wei Zhang | Ning Gao | Chaozheng Wang | Xinwei Wu | Yujie Wang | Wei He | Jinpeng Wang | Linlong Xu | Deyi Xiong
Findings of the Association for Computational Linguistics: EMNLP 2026
Ling Shi | Yuqin Dai | Ziyin Wang | Wei Zhang | Ning Gao | Chaozheng Wang | Xinwei Wu | Yujie Wang | Wei He | Jinpeng Wang | Linlong Xu | Deyi Xiong
Findings of the Association for Computational Linguistics: EMNLP 2026
LLMs are transforming customer service, yet evaluating service agents across diverse business scenarios remains costly and fragmented, as each new domain requires building a dedicated benchmark from scratch. Existing benchmarks rely on static, scenario-specific scripts with two fundamental limitations: ***Scenario Silos***, where each domain demands costly independent annotation with no shared evaluation structure, and ***Dialogue Dynamics Gap***, where fixed scripts fail to capture the full range of real user behaviors. We propose **SAGE** (**S**ervice **A**gent **G**raph-guided **E**valuation), to our knowledge the first scenario-extensible benchmark to jointly assess logical SOP compliance and conversational quality, abstracting any SOP as a directed graph and bridging Scenario Silos through a lightweight two-step extension mechanism. SAGE further enables dynamic multi-turn simulation via adversarial user agents to overcome the Dynamics Gap. Across six industrial scenarios representative of real-world customer service deployment, we evaluate 27 mainstream LLMs spanning both closed-source systems and open-source families, offering a fine-grained capability map to guide real-world model selection. Our experiments further reveal two systematic failure patterns shared across all tested models: while models generally succeed at recognizing user intent, they frequently struggle to follow complex business logic and execute the correct subsequent action; and they sustain a warm and empathetic conversational tone even when their underlying procedural reasoning has already failed. **Code and resources are available at: https://github.com/lingshi6565/SAGE-Bench.**
One Instruction Is a Benchmark: End-to-End Instruction-Following Evaluation with FlexBench
Yuchen Ge | Yizhao Sun | Wei He | Jinpeng Ou | Wenzhe Niu | Chengshun Shi | Huichuan Fan | Biao Ma | Shuo Huang | Xin Tian | Jiuchong Gao | Jinghua Hao | Renqing He
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yuchen Ge | Yizhao Sun | Wei He | Jinpeng Ou | Wenzhe Niu | Chengshun Shi | Huichuan Fan | Biao Ma | Shuo Huang | Xin Tian | Jiuchong Gao | Jinghua Hao | Renqing He
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Real-world deployments of large language models (LLMs) increasingly rely on user-provided instructions that are long, precise, and tailored to specific workflows. In such settings, aggregate leaderboard performance is often a weak proxy for a model’s ability to faithfully execute the particular instruction that a practitioner intends to deploy. To address this gap, we introduce FlexBench/FlexEval, an instruction-conditioned evaluation framework that instantiates a specialized test suite from a user-provided target instruction. Given a target instruction, FlexBench (i) derives a set of verifiable evaluation dimensions and (ii) generates an instruction-conditioned conversation corpus. FlexEval then aggregates dimension-level judgments into two complementary metrics: Coverage, which measures workflow progress, and Achievement, which measures conditional correctness. We evaluate the framework on nine multi-turn target instructions spanning diverse scenario categories and workflow topologies, across nine contemporary LLMs over 3,888 generated dialogues. The results show that model rankings and failure modes vary substantially across target instructions, highlighting the need for instruction-conditioned evaluation beyond fixed aggregate benchmarks. Our source code is publicly available at https://github.com/Coryyy/FlexBench.