Sihao Lin
Author directory2026
HEAL: Latent Head Alignment for Post-Training Quantization of Large Language Models
Mingzhe Yang | Sihao Lin | Ruibo Zhang | Xinyuan Zhu | Lei Chen | Weidong Chen | Xiaojun Chang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Mingzhe Yang | Sihao Lin | Ruibo Zhang | Xinyuan Zhu | Lei Chen | Weidong Chen | Xiaojun Chang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Post-training quantization (PTQ) has become a key approach for large language model (LLM) compression. Existing methods commonly adopt block-level reconstruction as a surrogate to calibrate each quantized Transformer block. However, our analysis shows that this loose objective can admit cancellation patterns, where the aggregated block-level reconstruction error is small even though the per-head errors remain large. We therefore revisit the PTQ formulation from a variational perspective. By treating per-head representations as latent variables and the block output as the observed variable, we find that conventional block-level reconstruction corresponds to optimizing only the observation-level term, while omitting a latent alignment term that captures head-level quantization error. Motivated by this finding, we propose HEAL, a Latent HEad ALignment framework that turns block-level reconstruction calibration into joint observation-latent calibration with stop-gradient head alignment. Experiments on LLaMA-2/3 and Qwen2.5 families under low-bit weight-activation quantization show that HEAL generally improves perplexity and zero-shot accuracy over block-level reconstruction baselines.