Group-Relative Policy Optimization (GRPO) is widely used to post-train LLM agents, but its trajectory-level advantage is ill-suited to long-horizon credit assignment. Reward-homogeneous rollout groups yield zero advantage; otherwise, one scalar uniformly scales every token, obscuring which decisions determine the outcome. We propose **TRACE-GRPO** (**T**eacher-**R**efined **A**dvantage **C**redit **E**stimation for GRPO), which converts teacher-LLM feedback into hierarchical credit. Teacher scores provide a sample-level fallback for zero-variance groups, reweight trajectory advantage at the turn level, and redistribute turn credit at the token level using the policy’s log-probability shift under a teacher critique. This yields dense training-time credit without token-level annotations or learned critics. We also introduce **CAPA** (Critique-Aware Paged Attention), which reuses trajectory KV caches and reduces the isolated critique-conditioned forward from roughly 2× a vanilla GRPO step to approximately 5% of a training step. On ALFWorld and ScienceWorld with Qwen2.5-7B-Instruct, TRACE-GRPO achieves 96.3% and 73.4% success, respectively, outperforming strong agent RL baselines. Ablations show cumulative gains from the three credit levels, and comparison with supervised fine-tuning on teacher-generated successes confirms that critique-guided policy optimization provides benefits beyond imitation. Code is available at [https://github.com/Lau-Jonathan/TRACE-GRPO](https://github.com/Lau-Jonathan/TRACE-GRPO).

In recent years, large language models (LLMs) have attracted significant interest from the research community due to their broad applicability in many language-oriented tasks, and are now widely used in numerous areas of production and daily life. One source of the powerful capabilities of LLMs is the massive scale of their pre-training dataset. However, these pre-training datasets contain many outdated, harmful, and personally sensitive information, which inevitably becomes memorized by LLM during the pre-training process. Eliminating this undesirable data is crucial for ensuring the model’s safety and enhancing the user experience. However, the cost of extensively cleaning the pre-training dataset and retraining the model from scratch is very high. In this work, we propose ULMR , a unlearning framework for LLMs , which first uses carefully designed prompts to rewrite the instructions in the specified dataset, and generate corresponding negative responses. Subsequently, to ensure that the model does not excessively deviate post-training, we perform model parameter averaging to preserve the performance of the original LLM. We conducted experiments on two public datasets, TOFU and RWKU, demonstrating that our method can effectively forget specified information while retaining the capabilities of the original LLM.

In recent years, conversational recommender systems (CRSs) have drawn a wide attention in the research community, which focus on providing high-quality recommendations to users via natural language conversations. However, due to diverse scenarios and data formats, existing studies on CRSs lack unified and standardized implementation or comparison. To tackle this challenge, we release an open-source toolkit CRSLab, which provides a unified and extensible framework with highly-decoupled modules to develop CRSs. Based on this framework, we collect 6 commonly used human-annotated CRS datasets and implement 19 models that include advanced techniques such as graph neural networks and pre-training models. Besides, our toolkit provides a series of automatic evaluation protocols and a human-machine interaction interface to evaluate and compare different CRS methods. The project and documents are released at https://github.com/RUCAIBox/CRSLab.