Reinforcement learning with verifiable rewards (RLVR) improves mathematical reasoning, but sparse sequence-level rewards make training heavily dependent on long, repeated rollout generation. On-policy distillation (OPD) makes these rollouts more informative by asking a larger teacher to score every student trajectory token by token, but this inserts large-teacher forward passes into every RL step and requires online resources beyond standard student-only RL. We propose SPEC-Distill, a speculative prefix distillation framework in which the teacher shapes rollout construction rather than defining an auxiliary reward. Instead of querying the teacher during training, SPEC-Distill runs it once offline to build a reusable pool of expert trajectories. During RL, the student treats these trajectories as speculative drafts, accepts a prefix under a lenience-controlled criterion, completes the response under its own policy, and is updated only by verifiable rewards with standard GRPO. Against OPD, this removes online teacher scoring; against standard autoregressive rollouts, accepted prefixes reduce the number of fresh student-generated tokens. In the primary non-think setting, SPEC-Distill improves Qwen3-4B from 66.1 to 73.2 average accuracy over GRPO, and outperforms OPD on the two scales. Compared with full-response expert reuse such as Luffy, SPEC-Distill reaches comparable accuracy on Qwen3-1.7B while reducing online rollout tokens by about 60% and rollout time from 10.90 to 7.06 hours. Analysis further shows that teacher prefixes are best used as an early scaffold, after which reusing the student’s own rollouts provides the strongest balance between expert guidance and on-policy exploration.