Reinforcement learning with verifiable rewards (RLVR) improves mathematical reasoning, but sparse sequence-level rewards make training heavily dependent on long, repeated rollout generation. On-policy distillation (OPD) makes these rollouts more informative by asking a larger teacher to score every student trajectory token by token, but this inserts large-teacher forward passes into every RL step and requires online resources beyond standard student-only RL. We propose SPEC-Distill, a speculative prefix distillation framework in which the teacher shapes rollout construction rather than defining an auxiliary reward. Instead of querying the teacher during training, SPEC-Distill runs it once offline to build a reusable pool of expert trajectories. During RL, the student treats these trajectories as speculative drafts, accepts a prefix under a lenience-controlled criterion, completes the response under its own policy, and is updated only by verifiable rewards with standard GRPO. Against OPD, this removes online teacher scoring; against standard autoregressive rollouts, accepted prefixes reduce the number of fresh student-generated tokens. In the primary non-think setting, SPEC-Distill improves Qwen3-4B from 66.1 to 73.2 average accuracy over GRPO, and outperforms OPD on the two scales. Compared with full-response expert reuse such as Luffy, SPEC-Distill reaches comparable accuracy on Qwen3-1.7B while reducing online rollout tokens by about 60% and rollout time from 10.90 to 7.06 hours. Analysis further shows that teacher prefixes are best used as an early scaffold, after which reusing the student’s own rollouts provides the strongest balance between expert guidance and on-policy exploration.

Retrieving code functions, classes or files relevant to a user query, bug report or feature request from large codebases is a fundamental challenge for Large Language Model (LLM)-based coding agents. Agentic approaches typically employ sparse methods like BM25 or dense embedding strategies to identify semantically relevant units. While dense embedding approaches can outperform BM25 by large margins, both ignore the graph-structured characteristics of the codebase. To address this, we propose SpIDER (Spatially Informed Dense Embedding Retrieval), a dense retrieval approach that integrates LLM-based reasoning with graph-based exploration of the codebase. We further introduce SpIDER-Bench, a graph-structured benchmark curated from SWEPolyBench, SWEBench-Verified and Multi-SWE-bench, spanning Python, Java, JavaScript and TypeScript repositories. SpIDER’s graph-based candidate expansion attaches a structural reason for inclusion to each surfaced function (its seed and the edge type linking them), making the candidate set auditable while keeping the retrieval budget fixed. The graph is built from per-repository syntax trees, so it can be constructed on-demand at the start of a developer session rather than precomputed offline. Empirical results show that SpIDER consistently improves dense retrieval Recall@20 across every language and benchmark in SpIDER-Bench: by at least 13% relative (+0.05 to +0.12 absolute) along containment edges, rising to at least 27% relative (+0.11 to +0.19 absolute) once call edges are explored.