Lijuan Yang
Author directory2026
DUPO: Dual-Mode Unified Policy Optimization for Tool-Using Agents
Zexin Hu | Mu Hu | Lijuan Yang | Yisen Wang | Xiaogang Zhu | KangJian Jiao | Yu Fu | Lihong Gu | Zhiyong Wang | Kun Hu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zexin Hu | Mu Hu | Lijuan Yang | Yisen Wang | Xiaogang Zhu | KangJian Jiao | Yu Fu | Lihong Gu | Zhiyong Wang | Kun Hu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Tool-using large language models (LLMs) have improved at producing executable API calls, but reliable tool use requires coordinating semantic intent with executable invocation, especially in multi-step settings. Existing tool-learning objectives and traces collapse these two roles into a single homogeneous generation stream, so the learning signal cannot distinguish a flawed plan from a flawed call. We propose Dual-Mode Unified Policy Optimization (DUPO), a typed trajectory optimization framework that represents each step with a PLAN span for the current subgoal and a TOOL span for the API invocation. DUPO follows a PLAN-to-TOOL rollout factorization, optimizes the two modes with mode-separated policy ratios, and introduces Chain-Aware Upstream Advantage, so that early planning errors, which propagate through the rest of the trajectory, receive stronger credit than later steps. To support this optimization, we introduce PlanArena, a human-annotated dataset of 20,646 trajectories across 7,941 scenarios with explicit PLAN–TOOL supervision. Experiments on BFCL, API-Bank, and Tau2 show that DUPO improves tool-use performance, with gains concentrated in multi-step settings. Our code and dataset will be released upon publication.