Zheng Yang
Author directory2026
Sample Learning State Guided Dynamic SFT–RL Integration Post-training
Zheng Yang | Yilin Liu | Zhuoran Li | Wenhao Gao | Bingwei Li | Haozhe Sun | Caixia Yuan | Fan Wan | Wu Chen | Xiaojie Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Zheng Yang | Yilin Liu | Zhuoran Li | Wenhao Gao | Bingwei Li | Haozhe Sun | Caixia Yuan | Fan Wan | Wu Chen | Xiaojie Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Integrating supervised fine-tuning (SFT) and reinforcement learning (RL) has become a common post-training paradigm for improving the reasoning capabilities of large language models. Existing methods typically rely on sequential SFT–RL pipelines or static loss weights, overlooking sample-level heterogeneity and evolving learning states. We propose Sample Learning State (SLS), a rollout-based characterization of each training sample using two complementary signals: sample mastery and exploration-trajectory dispersion. These signals estimate whether reward-driven updates are reliable for the core task and whether current rollouts provide informative contrast for RL optimization, respectively. Based on SLS, we design a state-aware coefficient that dynamically balances SFT and RL losses, increasing the RL weight only when the model demonstrates sufficient core-task competence and the sampled trajectories provide useful reward contrast. Experiments on Qwen2.5-Math models show that SLS achieves state-of-the-art results on four in-distribution and two out-of-distribution mathematical reasoning benchmarks. In multi-reward settings, SLS preserves core reasoning performance while mitigating reward hacking induced by auxiliary rewards.