Zhiyong Cao
Author directory2026
Disentangled Privileged States for Simulator-Assisted Multi-Turn Dialogue Policy Training
Zhiyong Cao | Dunqiang Liu | Sicheng Zhou | Lin Cong
Findings of the Association for Computational Linguistics: EMNLP 2026
Zhiyong Cao | Dunqiang Liu | Sicheng Zhou | Lin Cong
Findings of the Association for Computational Linguistics: EMNLP 2026
Simulator-assisted reinforcement learning is a practical way to train multi-turn dialogue agents, but policy optimization can be misled when simulated users provide inconsistent feedback across long interaction trajectories. In this paper, we focus on a simulator-side question: can structured latent state tracking improve measured simulator consistency and provide useful process supervision for dialogue policy training? Motivated by a standard model-based RL simulation-bound perspective, we introduce **DPS**, a framework that decouples task-conditioned privileged state tracking from surface utterance realization. These privileged states are hidden from the agent but used during training to derive dense, state-aware process rewards that complement sparse outcome signals. We also construct RecruitmentBench, a recruitment persuasion dialogue benchmark grounded in anonymized real-world logs. Experiments on RecruitmentBench and two public datasets use offline real-log-grounded and cross-simulator protocols to evaluate intrinsic simulator consistency and downstream simulator-assisted policy performance. Results indicate that DPS improves measured simulator fidelity and yields stronger fine-tuned policy performance in these controlled evaluation settings, while remaining competitive with strong prompt-based LLM agents. We further calibrate our LLM-based evaluator against human judgments on MATH-CHAT. Code is available at: https://github.com/TraceLite-AI/DPS.
MM-VeriRubric: Verifiable Evidence-Grounded Rubrics for Multimodal Reward Models
Zhiyong Cao | Bo Wang | Jintao Li | Shuxia Liu | Kairan Zhang | Aijia Wang | Yi-Liang Wei | Sicheng Zhou
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zhiyong Cao | Bo Wang | Jintao Li | Shuxia Liu | Kairan Zhang | Aijia Wang | Yi-Liang Wei | Sicheng Zhou
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Reward modeling is essential for multimodal alignment, yet most existing reward models only provide outcome reward, which is sparse and fails to capture human preferences. Rubrics can provide structured and fine grained evaluation signals, but building effective rubrics for multimodal tasks is challenging. In this work, we propose **MM-VeriRubric**, a framework for scalable construction of training data for multimodal rubric generation and rubric-based reward models. To better use visual evidence in images, we introduce task-aware visual clue extraction, which makes rubric generation more grounded in task-relevant visual information. MM-VeriRubric then generates evidence-grounded rubrics and constructs rubric-aligned rejected responses with controlled error dimensions, enabling the quality of each rubric to be verified through consistency checking. The verified data are used to train both a rubric generator and a rubric-based reward model. Experiments on multimodal reward model benchmarks and downstream Best of N tasks show that MM-VeriRubric improves preference judgment (+16.4% on VL-RewardBench) and provides more interpretable and diagnostic reward signals for multimodal alignment (+13.1% on MMMU-Pro using Best of 5 selection). Code is available at: https://github.com/TraceLite-AI/MM-VeriRubric.
SimRPD: Optimizing Recruitment Proactive Dialogue Agents through Simulator-Based Data Evaluation and Selection
Zhiyong Cao | Dunqiang Liu | Qi Dai | Haojun Xu | Huai Yuen Khor | Hao Wang | Huan He | Yafei Liu | Ke Ma | Ruqian Shi | Sicheng Zhou | Sijia Yao
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)
Zhiyong Cao | Dunqiang Liu | Qi Dai | Haojun Xu | Huai Yuen Khor | Hao Wang | Huan He | Yafei Liu | Ke Ma | Ruqian Shi | Sicheng Zhou | Sijia Yao
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)
Task-oriented proactive dialogue agents play a pivotal role in recruitment, particularly for steering conversations towards specific business outcomes, such as acquiring social-media contacts for private-channel conversion. Although supervised fine-tuning and reinforcement learning have proven effective for training such agents, their performance is heavily constrained by the scarcity of high-quality, goal-oriented domain-specific training data. To address this challenge, we propose SimRPD, a three-stage framework for training recruitment proactive dialogue agents. First, we develop a high-fidelity user simulator to synthesize large-scale conversational data through multi-turn online dialogue. Then we introduce a multi-dimensional evaluation framework based on Chain-of-Intention (CoI) to comprehensively assess the simulator and effectively select high-quality data, incorporating both global-level and instance-level metrics. Finally, we train the recruitment proactive dialogue agent on the selected dataset. Experiments in a real-world recruitment scenario demonstrate that SimRPD outperforms existing simulator-based data selection strategies, highlighting its practical value for industrial deployment and its potential applicability to other business-oriented dialogue scenarios.