Lin Cong
Author directory2026
Disentangled Privileged States for Simulator-Assisted Multi-Turn Dialogue Policy Training
Zhiyong Cao | Dunqiang Liu | Sicheng Zhou | Lin Cong
Findings of the Association for Computational Linguistics: EMNLP 2026
Zhiyong Cao | Dunqiang Liu | Sicheng Zhou | Lin Cong
Findings of the Association for Computational Linguistics: EMNLP 2026
Simulator-assisted reinforcement learning is a practical way to train multi-turn dialogue agents, but policy optimization can be misled when simulated users provide inconsistent feedback across long interaction trajectories. In this paper, we focus on a simulator-side question: can structured latent state tracking improve measured simulator consistency and provide useful process supervision for dialogue policy training? Motivated by a standard model-based RL simulation-bound perspective, we introduce **DPS**, a framework that decouples task-conditioned privileged state tracking from surface utterance realization. These privileged states are hidden from the agent but used during training to derive dense, state-aware process rewards that complement sparse outcome signals. We also construct RecruitmentBench, a recruitment persuasion dialogue benchmark grounded in anonymized real-world logs. Experiments on RecruitmentBench and two public datasets use offline real-log-grounded and cross-simulator protocols to evaluate intrinsic simulator consistency and downstream simulator-assisted policy performance. Results indicate that DPS improves measured simulator fidelity and yields stronger fine-tuned policy performance in these controlled evaluation settings, while remaining competitive with strong prompt-based LLM agents. We further calibrate our LLM-based evaluator against human judgments on MATH-CHAT. Code is available at: https://github.com/TraceLite-AI/DPS.