Group Relative Policy Optimization (GRPO) and its variants have proven effective for reinforcement learning with verifiable rewards through outcome-based feedback. Incorporating intermediate credit, however, remains challenging. We show that, under practical sampling budgets, token-level Monte Carlo estimates at early reasoning positions have high variance, which can cause frequent advantage-sign errors after group-relative normalization and yield wrong-direction token-level gradient contributions. At later positions, these estimates increasingly resemble the terminal outcome reward and provide limited additional information.

We propose SCRoll, a rollout-side module that augments group-relative policy optimizers with randomized single-boundary segment-level rewards while leaving the downstream optimizer interface unchanged. SCRoll uses a skip-connected two-stream rollout. An upstream stream generates an early-stopped reasoning segment s, while a downstream stream samples G continuations conditioned on [s,q], which places the segment before the original problem. This construction retains the segment as preceding reasoning context while mitigating the reward homogenization associated with pure-prefix continuation. The same downstream group supplies terminal outcomes for the standard group-relative objective and a Monte Carlo aggregate used as the upstream segment reward, avoiding a separate objective-specific rollout group. The upstream segment is optimized with a sample-efficient single-stream objective.

Across Qwen2.5-Math-7B and Llama-3.2-3B-Instruct, pairing SCRoll with four representative group-relative optimizers improves performance in most evaluated settings. The clearest gains occur with CISPO, with smaller positive average gains for GRPO and DAPO and marginal gains for GSPO. A decoupled two-process baseline can achieve comparable performance with substantially more generated tokens, while SCRoll performs better under a matched generated-token budget, supporting the sample-efficiency benefit of rollout reuse.