Dynamic sampling is a technique employed in reinforcement Learning for Large Language Models to mitigate training instability, often caused by the gradient-decreasing problem. However, existing dynamic sampling approaches are predominantly designed for tasks with binary rewards (success/failure) and struggle to adapt to the complex reward structures of tool learning. Such task involves interdependent sub-tasks of varying difficulty and yields fine-grained, multi-faceted rewards. To bridge this gap, we introduce Dynamic Sampling with Curriculum Learning (DSCL), an algorithm tailored for the intricate dynamics of tool learning. DSCL integrates two core components: a Reward-Based Dynamic Sampler, which leverages multi-dimensional reward statistics to prioritize high-value training data; and a Task-Based Dynamic Curriculum Learning method to overcome the credit assignment problem caused by the fine-grained rewards. Extensive experiments on widely used tool learning benchmarks demonstrate the efficacy of our approach. DSCL significantly improves model performance, outperforming strong baselines by 4.75% on the BFCL V3 dataset and 4.02% on the API-Bank dataset.

Large Language Models (LLMs) have demonstrated significant advancements in various fields, notably in Role-Playing Conversational Agents (RPCAs). However, when confronted with role-specific professional inquiries, LLMs-based RPCAs tend to underperform due to their excessive emphasis on the conversational abilities of characters rather than effectively invoking and integrating relevant expert knowledge. This often results in inaccurate responses. We refer to this phenomenon as the “Knowledge Misalignment” which underscores the limitations of RPCAs in integrating expert knowledge. To mitigate this issue, we have introduced an Anchoring-Guidance Fine-Tuning (AnGFT) Framework into the RPCAs’ training process. This involves initially linking the Anchoring-Based System Prompt (ASP) with the LLM’s relevant expert domains through diverse prompt construction strategies and supervised fine-tuning (SFT). Following the role-play enriched SFT, the integration of ASP enables LLMs to better associate with relevant expert knowledge, thus enhancing their response capabilities in role-specific expert domains. Moreover, we have developed four comprehensive metrics—helpfulness, thoroughness, credibility, and feasibility—to evaluate the proficiency of RPCAs in responding to professional questions. Our method was tested across four professional fields, and the experimental outcomes suggest that the proposed AnGFT Framework substantially improves the RPCAs’ performance in handling role-specific professional queries, while preserving their robust role-playing abilities.