Huangxun Chen
Author directory2026
TurnFlow: Benchmarking User-State Reasoning in Multi-Turn LLM Agents via Scene-Guided User Simulation
Zhixun Chen | Ming Li | Anjie Liu | Xiaoyuan Zhang | Quanyu Dai | Huangxun Chen
Findings of the Association for Computational Linguistics: EMNLP 2026
Zhixun Chen | Ming Li | Anjie Liu | Xiaoyuan Zhang | Quanyu Dai | Huangxun Chen
Findings of the Association for Computational Linguistics: EMNLP 2026
Large Language Models (LLMs) are increasingly deployed through conversational interfaces, yet strong single-turn performance does not imply reliability in multi-turn setting. Multi-turn assistance requires a model to maintain an evolving representation of user requirements and to reason over that representation before complying with later requests. Failures can therefore be subtle: models may fail to recognize a changed constraint or hidden conflict, or recognize it without enforcing it in the final response. Existing benchmarks primarily test fully specified prompts, long-context recall, or controlled domain environments, leaving open-domain requirement reconciliation undermeasured. We introduce TurnFlow, an interactive benchmark for evaluating whether models preserve, revise, and act on user requirements across turns. TurnFlow combines adaptive user simulation with verifiable judging, allowing conversations to unfold naturally while keeping hidden requirements and pass/fail criteria controlled. In our main evaluation, twelve current LLMs achieve only 27.5%–51.0% aggregate pass rate on 490 scenarios. The failures show that current LLMs can remain fluent and cooperative while losing the global state of the conversation, exposing a gap between conversational helpfulness and robust requirement reconciliation.