Kaiwen Zheng
Author directoryOther people with similar names: Kaiwen Zheng
2026
Target-Aware Calibration Data Selection for Preserving Uncertainty in Quantized Language Models
Zhen Yang | Sizai Hou | Kaiwen Zheng | Yaofang Liu | Liang He | Yixuan Chen | Kangning Cui
Findings of the Association for Computational Linguistics: EMNLP 2026
Zhen Yang | Sizai Hou | Kaiwen Zheng | Yaofang Liu | Liang He | Yixuan Chen | Kangning Cui
Findings of the Association for Computational Linguistics: EMNLP 2026
Quantization is widely used to deploy large language models, but its effect on uncertainty behavior, such as confidence, margins, and abstention, is rarely treated as a primary objective. We frame calibration-data selection for quantization as a *target-dependent uncertainty-preservation problem*. Different deployments emphasize different regions of the input distribution, yet prior work mainly optimizes accuracy-oriented compression metrics or adjusts scores after quantization. We formalize this goal with distributional and boundary preservation risks, and provide a simple mixture-mismatch argument explaining why no single calibration recipe should be expected to fit all targets. We introduce Doubt-Preserving Quantization (DPQ), a lightweight pre-quantization recipe family that uses full-precision predictions to construct target-aligned calibration mixtures of high-doubt examples and generic anchors. Across 8 language models, 9 NLP benchmarks, and 22 comparison methods, the leading recipe changes with the preservation target: DPQ-r75 leads on SQuAD2 answerability-boundary preservation, while broad multiple-choice QA favors a different set of milder or single-signal recipes, with confidence-only leading and variants such as entropy-only and DPQ-r50 outperforming the boundary-heavy DPQ-r75. These results show that calibration data should be selected for the specific full-precision score behavior a deployment needs to preserve, rather than treated as a fixed quantization detail. Code is available at https://github.com/xi-xiaoran/DPQ.