Lin Ma
Author directoryOther people with similar names: Lin Ma, Lin Ma
Unverified author pages with similar names: Lin Ma
2026
Process Rewarding via Solution-Space Alignment
ChenZhuo Zhao | Yue Huang | Pu Zhao | Fangkai Yang | Lu Wang | Lin Ma | Zhi Yang | Youling Huang | Xinda Wang | Junting Lu | Qingwei Lin | Saravan Rajmohan | Dongmei Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
ChenZhuo Zhao | Yue Huang | Pu Zhao | Fangkai Yang | Lu Wang | Lin Ma | Zhi Yang | Youling Huang | Xinda Wang | Junting Lu | Qingwei Lin | Saravan Rajmohan | Dongmei Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Reinforcement learning with verifiable or outcome-level rewards has recently become an effective post-training paradigm for improving the reasoning abilities of large language models, with GRPO-style optimization achieving strong results on mathematical and other verifiable tasks. However, for multi-step reasoning, outcome-level supervision remains intrinsically sparse: a trajectory that contains partially correct intermediate derivations but makes a late mistake is still treated as a complete failure. As a result, useful reasoning prefixes are penalized together with erroneous continuations, and informative failed trajectories are underutilized. PRMs provide denser step-level feedback, but they often rely on costly process annotation or auxiliary process verifiers, and optimizing local process rewards alone may fail to preserve the global objective of final correctness, leaving room for reward misspecification and reward hacking during reinforcement fine-tuning. We propose PRSA(Process Rewarding via Solution-Space Alignment), a simple framework that bridges outcome supervision and process supervision without requiring human step-level labels. PRSA automatically extracts intermediate-value steps from model outputs and reference solutions, aligns them within the solution space under equivalence and order constraints to estimate reasoning progress, and uses this signal for both reward shaping and token-level prefix-suffix credit assignment. In this way, PRSA preserves the global anchor of final-answer correctness while providing denser and better localized learning signals for partially correct trajectories.