Group-based reinforcement learning typically relies on coarse outcome-level signals, such as group accuracy or variance, for data selection or replay. Yet in multi-turn tool-use rollout groups, these signals miss structural differences between successful and failed trajectories: a failure may diverge into an unrelated tool chain or differ from a successful trajectory only at a local decision. We therefore introduce Group Structural Proximity (GSP), which measures failed-to-successful rollout alignment within a group using tool dependency graphs induced by tool calls and argument flow. A controlled diagnostic study with outcome-statistics-matched subsets further shows that GSP captures distinct learning dynamics: high-GSP data drives localized policy refinement, while low-GSP data induces broader policy movement and higher-entropy exploration. This positions GSP as an effective metric on the exploration–exploitation tradeoff. Building on this insight, we introduce Proximity-Guided Policy Optimization (PGPO), which considers replay buffers with high- and low- GSP under an Exploration-to-Refinement Schedule (ERS) for group-based training. On 𝜏²-Bench and ACEBench Agent, PGPO improves sample efficiency and downstream tool-use performance, demonstrating the benefit of structure-aware group comparison in multi-turn tool-use reinforcement learning.

Cold-start supervised fine-tuning (SFT) is a critical initialization step before reinforcement learning from verifiable rewards (RLVR). Yet existing cold-start data selection methods largely inherit criteria from supervised instruction tuning—sample quality, difficulty, perplexity, or diversity—implicitly assuming that a stronger SFT warm start is also a better starting point for RL. We argue that this assumption does not always hold in the SFT-to-RL pipeline: cold-start SFT not only reduces supervised loss but also shapes the initial update geometry that downstream reward optimization inherits. Building on this observation, we propose RL-Proxy-Coverage. Our method estimates a low-rank, reward-relevant gradient subspace from a small set of representative RL anchor prompts and verifier rewards, and selects, as the cold-start subset, those candidate SFT examples whose gradients provide non-redundant coverage of this subspace. In controlled mathematical-reasoning experiments, RL-Proxy-Coverage outperforms representative SFT-centric, hard-diverse, and reward-aware baselines at the early-to-mid RL checkpoints. Our findings suggest a new view of cold-start data: it should be regarded not merely as supervised examples for improving SFT, but as an initialization resource that shapes the starting point of downstream reward optimization.

In this paper, we present a novel pipeline for the XLLM Shared Task-III: Large Language Model for Structural Reasoning (LLM-SR). Our pipeline addresses key challenges in automatic process-reward training data construction, such as high manual annotation costs, limited accuracy of large models in structured data processing, and dependency on auxiliary information for validation. To overcome these limitations, we first decompose the construction process into extraction and validation phases. Leveraging model-generated annotations, we produce pseudo-labeled data and iteratively refine model performance. Second, by analyzing structured data patterns, we encode structural constraints into a rule-based module and fine-tune the model with Gradient Reward Policy Optimization (GRPO), significantly improving structured data extraction success rates. Finally, we train the model to generate critical responses that assess evidence-conclusion relationships, thus enhancing validation reliability. Experimental results demonstrate that our pipeline outperforms models with an order of magnitude more parameters and achieves the first position on the task.

The few-shot tasks require the model to have the ability to generalize from a few samples. However, due to the lack of cognitive ability, the current works cannot fully utilize limited samples to expand the sample space and still suffer from overfitting issues. To address the problems, we propose a LLM-Augmented Unsupervised Contrastive Learning Framework (LA-UCL), which introduces a cognition-enabled Large Language Model (LLM) for efficient data augmentation, and presents corresponding contrastive learning strategies. Specifically, in the self-augmented contrastive learning module, we construct a retrieval-based in-context prompt scheme by retrieving similar but different category data from the original samples, guiding the LLM to generate more discriminative augmented data. Then, by designing group-level contrastive loss to enhance the model’s discriminative ability. In the external-augmented contrastive learning module, we utilize web knowledge retrieval to expand the sample space and leverage LLM to generate more diverse data, and introduce sample-level contrastive loss for unlabeled data to improve the model’s generalization. Experimental results on six datasets show that our model exceeds the baseline models.