Reinforcement learning with verifiable rewards (RLVR) improves the reasoning ability of large language models (LLMs), but its standard form relies on ground-truth answers or external verifiers that are often unavailable in unlabeled test-time settings. Label-free RLVR therefore estimates pseudo rewards from model outputs. Existing methods typically use these pseudo rewards to optimize either answers or model parameters. The former cannot preserve feedback across queries, while the latter risks committing noisy signals to model parameters. This raises a central question: what should label-free rewards optimize? We propose Test-Time Memory Learning (TTML), a label-free RLVR framework that shifts the optimization target from answers or parameters to an external memory bank. TTML consists of three components: memory policy rollout selects useful memories and executes memory-augmented generation for the current query, Bayesian value update turns consensus rewards into memory value updates, and memory bank expansion extracts reusable experience from test-time trajectories. Across six benchmarks and three backbone models, TTML achieves the best Avg@8 performance over strong label-free baselines. Empirical analyses further suggest that converting label-free feedback into revisable memory values and reusable memory entries is more effective overall than using it only for single-query selection or direct parameter updates.

Reinforcement learning with verifiable rewards (RLVR) enhances reasoning in large language models (LLMs) but typically depends on human-annotated answers or curated reward specifications. Label-free alternatives such as majority voting or LLM-as-a-Judge remove this cost but introduce false positives that destabilize training. We introduce JURY-RL, a label-free RLVR framework that decouples answer proposal from reward disposal: rollouts matching the plurality-voted answer receive positive reward only when that answer is verified in Lean. When verification is inconclusive, ResZero (Residual-Zero) discards the unverified plurality and redistributes a zero-mean, variance-preserving signal over the residual answers, keeping the optimization gradient stable without reinforcing unverifiable consensus. Across three backbones trained on math data, JURY-RL outperforms label-free baselines on mathematical reasoning, transfers to code generation and general benchmarks, and surpasses a majority-vote baseline in the test-time RL (TTRL) setting; it matches supervised ground-truth training on pass@1 and beats it on pass@k and response diversity.

Reinforcement learning with verifiable rewards (RLVR) has recently advanced the reasoning capabilities of large language models (LLMs). We investigate RLVR from a sample-centric perspective and introduce LPPO (Learning-Progress and Prefix-guided Optimization), a framework of progressive optimization techniques. Our work addresses a critical question: how to best leverage a small set of trusted, high-quality demonstrations, rather than simply scaling up data volume. First, motivated by how hints aid human problem-solving, we propose prefix-guided sampling, an online data augmentation method that incorporates partial solution prefixes from expert demonstrations to guide the policy, particularly for challenging instances. Second, inspired by how humans focus on important questions aligned with their current capabilities, we introduce learning-progress weighting, a dynamic strategy that adjusts each training sample’s influence based on model progression. We estimate sample-level learning progress via an exponential moving average of per-sample pass rates, promoting samples that foster learning and de-emphasizing stagnant ones. Experiments on mathematical-reasoning benchmarks demonstrate that our methods outperform strong baselines, yielding faster convergence and a higher performance ceiling, with these gains proving robust across diverse model architectures, scales, and reinforcement learning optimizers.