Pedro Gimenes
Author directory2026
Scaling Laws for Mixed Quantization
Zeyu Cao | Yao Lai | Boyang Gu | Cheng Zhang | Pedro Gimenes | Jianqiao Lu | Xitong Gao | Jianyi Cheng | Robert D. Mullins | Yiren Zhao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zeyu Cao | Yao Lai | Boyang Gu | Cheng Zhang | Pedro Gimenes | Jianqiao Lu | Xitong Gao | Jianyi Cheng | Robert D. Mullins | Yiren Zhao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Post-training quantization (PTQ) of large language models (LLMs) reduces memory usage and accelerates inference while maintaining accuracy. However, it remains unclear how to select a quantization scheme that achieves target accuracy across different model scales. We introduce two parameters to characterize this: the quantization ratio Qr (proportion of parameters quantized to low precision) and the block size Qb (number of parameters sharing a scaling factor). Based on extensive experiments across multiple LLM families and scales, we propose a unified scaling law that predicts quantization-induced loss degradation as a function of Qr, Qb, and model size. Our scaling law reveals two practical implications: (1) larger models tolerate higher Qr, making mixed-precision increasingly effective at scale; (2) small Qb, commonly used in mixed precision, becomes unnecessary for large models, simplifying hardware design without sacrificing accuracy. These findings provide a principled recipe for quantizing LLMs in practical deployment.