Pedro Gimenes

Author directory

2026

Post-training quantization (PTQ) of large language models (LLMs) reduces memory usage and accelerates inference while maintaining accuracy. However, it remains unclear how to select a quantization scheme that achieves target accuracy across different model scales. We introduce two parameters to characterize this: the quantization ratio Qr (proportion of parameters quantized to low precision) and the block size Qb (number of parameters sharing a scaling factor). Based on extensive experiments across multiple LLM families and scales, we propose a unified scaling law that predicts quantization-induced loss degradation as a function of Qr, Qb, and model size. Our scaling law reveals two practical implications: (1) larger models tolerate higher Qr, making mixed-precision increasingly effective at scale; (2) small Qb, commonly used in mixed precision, becomes unnecessary for large models, simplifying hardware design without sacrificing accuracy. These findings provide a principled recipe for quantizing LLMs in practical deployment.