Criterion-Activated Preference Synthesis: Attribution-Preserving Preference Data from Explicit Standards

Zian Wang, Pu Lu, Caishuang Huang, Wuwenxi, Meng Zhou, Xiku Du, Yang Qiao


Abstract
Direct Preference Optimization (DPO) is notoriously sensitive to attribution ambiguity: when chosen and rejected responses differ along multiple entangled dimensions, the model cannot identify which behavior the preference label teaches. We propose Criterion-Activated Preference Synthesis (CAPS), a criterion-first framework that turns explicit quality standards into attribution-verified preference pairs. CAPS synthesizes contexts and responses from a target criterion, then applies dual-signal scoring: a rubric-bound judge measures criterion compliance, while a blind general judge measures holistic quality. Two filters close the attribution loop: the Empirical Activation Test (EAT) keeps criterion-context pairs where satisfying the criterion improves blind quality, and the Attribution Ratio (AR) keeps pairs whose quality gap is dominated by the target criterion. Instantiated with general rubrics, medical rubrics, and Chinese legal statutes, CAPS shows strong data efficiency and domain transfer. With 20k general synthetic pairs, it outperforms 35k human-annotated HelpSteer3 pairs on IFEval, IFBench, AlpacaEval, and Creative Writing v3. In domain settings, CAPS improves HealthBench with 2k medical pairs and improves Qwen3-8B/14B by +4.9 to +6.6 points on legal benchmarks using only public statutes, without human preference labels. We additionally construct the CAPS-Legal preference dataset.
Anthology ID:
2026.emnlp-main.1213
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
27719–27738
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1213/
DOI:
Bibkey:
Cite (ACL):
Zian Wang, Pu Lu, Caishuang Huang, Wuwenxi, Meng Zhou, Xiku Du, and Yang Qiao. 2026. Criterion-Activated Preference Synthesis: Attribution-Preserving Preference Data from Explicit Standards. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 27719–27738, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Criterion-Activated Preference Synthesis: Attribution-Preserving Preference Data from Explicit Standards (Wang et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1213.pdf