Peirou Liang
Author directory2026
Consistency-Calibrated Advantage Estimation for Label-Free Reinforcement Learning
Meng Yang | Yongle Zhao | Zhiyang Zhang | Xiaorui Jiang | Peirou Liang | Furui Chen | Yin Xie | Ziyong Feng
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Meng Yang | Yongle Zhao | Zhiyang Zhang | Xiaorui Jiang | Peirou Liang | Furui Chen | Yin Xie | Ziyong Feng
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Reinforcement learning from unlabeled prompts offers a promising path to improving LLM reasoning without curated supervision or verifiable rewards. However, existing label-free methods typically rely on either intrinsic self-certainty or extrinsic answer agreement, both of which can become unreliable in hard reasoning regimes: confident trajectories can be wrong, while majority answers can drift toward stable yet incorrect attractors. We propose CCAE, a verifier-free and parameter-free advantage estimator that formulates label-free GRPO as a multi-signal advantage estimation problem. CCAE decomposes the advantage into a base self-certainty term, an intrinsic evidence factor, and an extrinsic agreement factor, with their interaction adaptively calibrated by a group-level proxy for agreement reliability, without introducing any reward model, verifier, or trainable parameters. On MATH-Train, CCAE improves over the strongest unlabeled baseline by 5.0 and 4.3 points on Qwen3-4B/8B-Base, respectively, matching supervised GRPO. On the harder DAPO-14k-MATH, CCAE remains stable while prior unlabeled methods collapse, outperforming their best pre-collapse checkpoints by up to 11.1 points. Results on Natural Reasoning further suggest that CCAE remains effective beyond math-only training. Code is available at: [https://github.com/tifing/CCAE](https://github.com/tifing/CCAE).
2025
Efficient Data Labeling by Hierarchical Crowdsourcing with Large Language Models
Haodi Zhang | Junyu Yang | Jinyin Nie | Peirou Liang | Kaishun Wu | Defu Lian | Rui Mao | Yuanfeng Song
Proceedings of the 31st International Conference on Computational Linguistics
Haodi Zhang | Junyu Yang | Jinyin Nie | Peirou Liang | Kaishun Wu | Defu Lian | Rui Mao | Yuanfeng Song
Proceedings of the 31st International Conference on Computational Linguistics
Large language models (LLMs) have received lots of attention for their impressive performance in in-context dialogues and their potential to revolutionize service industries with a new business model, Model-as-a-Service (MaaS). Automated data labeling is a natural and promising service. However, labeling data with LLMs faces two main challenges: 1) the labels from LLMs may contain uncertainty, and 2) using LLMs for data labeling tasks can be prohibitively expensive, as the scales of datasets are usually tremendous. In this paper, we propose a hierarchical framework named LMCrowd that leverages multiple LLMs for efficient data labeling under budget constraints. The proposed LMCrowd framework first aggregates labels from multiple freely available LLMs, and then employs a large, paid MaaS LLM for relabeling selected instances. Furthermore, we formalize the core process as an optimization problem, aiming to select the optimal set of instances for relabeling by the MaaS LLM, given the current belief state. Extensive experimental evaluations across various real-world datasets demonstrate that our framework outperforms human labelers and GPT-4 in terms of both accuracy and efficiency.