Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising framework for optimizing large language models in reasoning tasks. However, existing RLVR algorithms focus on different granularities, and each has complementary strengths and limitations. Group Relative Policy Optimization (GRPO) updates the policy with token-level importance ratios, which preserves fine-grained credit assignment but often suffers from high variance and instability. In contrast, Group Sequence Policy Optimization (GSPO) applies single sequence-level importance ratios across all tokens in a response that better matches sequence-level rewards, but sacrifices token-wise credit assignment. In this paper, we propose Dynamic Hybrid Policy Optimization (DHPO) to bridge GRPO and GSPO within a single clipped surrogate objective. DHPO combines token-level and sequence-level importance ratios using weighting mechanisms. We explore two variants of the mixing mechanism, including an averaged mixing and an entropy-guided mixing. To further stabilize training, we employ a branch-specific clipping strategy that constrains token-level and sequence-level ratios within separate trust regions before mixing, preventing outliers in either branch from dominating the update. Across seven challenging mathematical reasoning benchmarks, experiments on both dense and MoE models from the Qwen3 series show that DHPO consistently outperforms GRPO and GSPO. Our code is publicly available at https://github.com/XMUDeepLIT/DHPO.

Reinforcement learning with verifiable rewards (RLVR) improves mathematical reasoning, but sparse sequence-level rewards make training heavily dependent on long, repeated rollout generation. On-policy distillation (OPD) makes these rollouts more informative by asking a larger teacher to score every student trajectory token by token, but this inserts large-teacher forward passes into every RL step and requires online resources beyond standard student-only RL. We propose SPEC-Distill, a speculative prefix distillation framework in which the teacher shapes rollout construction rather than defining an auxiliary reward. Instead of querying the teacher during training, SPEC-Distill runs it once offline to build a reusable pool of expert trajectories. During RL, the student treats these trajectories as speculative drafts, accepts a prefix under a lenience-controlled criterion, completes the response under its own policy, and is updated only by verifiable rewards with standard GRPO. Against OPD, this removes online teacher scoring; against standard autoregressive rollouts, accepted prefixes reduce the number of fresh student-generated tokens. In the primary non-think setting, SPEC-Distill improves Qwen3-4B from 66.1 to 73.2 average accuracy over GRPO, and outperforms OPD on the two scales. Compared with full-response expert reuse such as Luffy, SPEC-Distill reaches comparable accuracy on Qwen3-1.7B while reducing online rollout tokens by about 60% and rollout time from 10.90 to 7.06 hours. Analysis further shows that teacher prefixes are best used as an early scaffold, after which reusing the student’s own rollouts provides the strongest balance between expert guidance and on-policy exploration.