Xiaohan He
Author directory2026
RewardCode: Training Generalist Code Reward Model via Pairwise Reinforcement Learning
Songtao Huang | Shiyang Feng | Xiaohan He | Xiangchao Yan | Bo Zhang | Lei Bai
Findings of the Association for Computational Linguistics: EMNLP 2026
Songtao Huang | Shiyang Feng | Xiaohan He | Xiangchao Yan | Bo Zhang | Lei Bai
Findings of the Association for Computational Linguistics: EMNLP 2026
Reward modeling aligns language models with human preferences, yet coding tasks, requiring strict logic and syntax, pose challenges due to their domain diversity. In this paper, we propose RewardCode, a generalist code reward model trained via a two-stage pipeline, enabling scalable and precise evaluation through principle-based scoring and unit test generation. Specifically, the first stage combines Structured Fine-Tuning and Rejective Fine-Tuning to distill cross-domain summarization and code evaluation capabilities from advanced LLMs. The second stage introduces Pairwise-GRPO, a reinforcement learning method that uses preference pairs to improve solution discrimination, while ensuring the generation of consistent and verifiable unit tests. Further, to train a cross-domain code reward model, we construct CodePair-18K, a dataset of verifiable code preference pairs and executable unit tests. Experiments on multiple benchmarks show that RewardCode outperforms strong baselines in evaluation accuracy, verifying its effectiveness in advancing general-purpose code LLMs.