Mingzhe Yang

Author directory

2026

Post-training quantization (PTQ) has become a key approach for large language model (LLM) compression. Existing methods commonly adopt block-level reconstruction as a surrogate to calibrate each quantized Transformer block. However, our analysis shows that this loose objective can admit cancellation patterns, where the aggregated block-level reconstruction error is small even though the per-head errors remain large. We therefore revisit the PTQ formulation from a variational perspective. By treating per-head representations as latent variables and the block output as the observed variable, we find that conventional block-level reconstruction corresponds to optimizing only the observation-level term, while omitting a latent alignment term that captures head-level quantization error. Motivated by this finding, we propose HEAL, a Latent HEad ALignment framework that turns block-level reconstruction calibration into joint observation-latent calibration with stop-gradient head alignment. Experiments on LLaMA-2/3 and Qwen2.5 families under low-bit weight-activation quantization show that HEAL generally improves perplexity and zero-shot accuracy over block-level reconstruction baselines.