2026

LLMs are transforming customer service, yet evaluating service agents across diverse business scenarios remains costly and fragmented, as each new domain requires building a dedicated benchmark from scratch. Existing benchmarks rely on static, scenario-specific scripts with two fundamental limitations: ***Scenario Silos***, where each domain demands costly independent annotation with no shared evaluation structure, and ***Dialogue Dynamics Gap***, where fixed scripts fail to capture the full range of real user behaviors. We propose **SAGE** (**S**ervice **A**gent **G**raph-guided **E**valuation), to our knowledge the first scenario-extensible benchmark to jointly assess logical SOP compliance and conversational quality, abstracting any SOP as a directed graph and bridging Scenario Silos through a lightweight two-step extension mechanism. SAGE further enables dynamic multi-turn simulation via adversarial user agents to overcome the Dynamics Gap. Across six industrial scenarios representative of real-world customer service deployment, we evaluate 27 mainstream LLMs spanning both closed-source systems and open-source families, offering a fine-grained capability map to guide real-world model selection. Our experiments further reveal two systematic failure patterns shared across all tested models: while models generally succeed at recognizing user intent, they frequently struggle to follow complex business logic and execute the correct subsequent action; and they sustain a warm and empathetic conversational tone even when their underlying procedural reasoning has already failed. **Code and resources are available at: https://github.com/lingshi6565/SAGE-Bench.**
Large Language Models (LLMs) have demonstrated remarkable capabilities in open-domain dialogues. However, their performance in service dialogues remains suboptimal, as these require agents to guide users toward specific business objectives while dynamically tracking states and adapting strategies. This gap stems from the scarcity of high-quality training data and the difficulty in simulating authentic, goal-oriented user behaviors. We propose SEAD (Self-Evolving Agent for Service Dialogue), a framework that enables agents to learn effective strategies without large-scale human annotations. SEAD decouples user modeling into two components: a Profile Controller that generates diverse user states to manage training curriculum, and a User Simulator that focuses on realistic role-playing. This design ensures the environment provides adaptive training scenarios rather than acting as an unfair adversary.

2016