Bolun Sun
Author directory2026
SCOPE: Stepwise Calibration of Reasoning Confidence with Outcome-Grounded Prefix Evidence
Huaqin Zhao | Yifan Zhou | Bolun Sun | Chao Cao | Yiwei Li | Zhaojun Ding | Lu Zhang | Tianming Liu | Jin Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Huaqin Zhao | Yifan Zhou | Bolun Sun | Chao Cao | Yiwei Li | Zhaojun Ding | Lu Zhang | Tianming Liu | Jin Lu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Reinforcement learning with verifiable rewards (RLVR) improves language-model reasoning using outcome-level supervision, but this supervision is temporally coarse: it indicates whether a trajectory ends with a correct answer, without revealing when the reasoning becomes reliable or silently fails. Existing verbalized-confidence methods typically calibrate a single confidence score after the final answer, leaving intermediate uncertainty unobserved. We propose SCOPE (Stepwise Calibration with Outcome-grounded Prefix Evidence), which trains reasoning models to emit confidence after each reasoning step. SCOPE interprets step confidence as the model’s prefix-conditioned belief in the gold answer: how likely the gold answer is under the reasoning prefix generated so far. During training, SCOPE constructs a soft target by scoring the gold answer conditioned on each reasoning prefix under the current policy. It then distills this signal with a two-stage masked RL objective: the first stage stabilizes the output format and final-answer accuracy, while the second stage performs calibration-only masked updates by applying stepwise and final-confidence rewards only to the numerical tokens inside the emitted confidence spans. Experiments on mathematical reasoning benchmarks show that SCOPE improves stepwise and final-confidence calibration while preserving final-answer reasoning performance. We further show that the resulting confidence signals support downstream inference-time control, enabling more reliable Learn-Then-Test risk control and stronger confidence-weighted aggregation.
CAPC-CG: A Large-Scale, Expert-Directed LLM-Annotated Corpus of Adaptive Policy Communication in China
Bolun Sun | Charles Chang | Yuen Yuen Ang | Ruotong Mu | Yuchen Xu | Zhengxin Zhang | Pingxu Hao
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Bolun Sun | Charles Chang | Yuen Yuen Ang | Ruotong Mu | Yuchen Xu | Zhengxin Zhang | Pingxu Hao
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
We introduce CAPC-CG, the Chinese Adaptive Policy Communication (Central Government) Corpus, the first open dataset of Chinese policy directives annotated with a five-color typology of policy signals, capturing clarity and ambiguity, grounded in the theory of adaptive policy communication. Spanning 1949–2023, this corpus includes laws, regulations, and rules issued by Chinese central authorities, segmented into 3.3 million paragraph units. We further propose and validate an expert-directed LLM annotation method that integrates codebook design, structured training, a two-step workflow, and LLM-based scaling. Alongside the corpus, we release metadata and a gold-standard labeled set developed by trained coders. Inter-annotator agreement achieves a Fleiss’ kappa of κ = 0.86 on directive labels, indicating high reliability. We provide baseline classification results with several large language models (LLMs), together with our codebook, and describe patterns from the data. This release enables downstream tasks and multilingual NLP research in communication strategies under complexity and uncertainty.