Extreme low-bit quantization-aware training (QAT) is often unstable due to synchronized bin transitions, which can trigger catastrophic loss spikes. While prior works rely on scale-side controls to mitigate this fragility, we demonstrate that restricting pivot movement alone is insufficient. Instead, we identify a critical weight-side state governing low-bit QAT stability: pivot-proximal residence, where latent weights persistently linger near quantization pivots. We formalize this duration as dwell time and show that long dwell times accumulate a large, vulnerable population of weights prone to synchronized mass flips. To resolve this vulnerability, we propose Pivot Gradient Scaling (PGS) as a principled modification to the Straight-Through Estimator (STE) that leaves the forward pass intact and selectively amplifies latent-weight gradients within a localized pivot band. Crucially, we provide a mathematical guarantee proving that this sign-preserving amplification increases pivot-proximal mobility and systematically shortens expected dwell time under both SGD and Adam. Exhaustive empirical evaluations across Llama 3.2 and Qwen3 model families under full-training regimes up to 30 billion tokens demonstrate that PGS structurally stabilizes optimization, attenuates transition bursts, and delivers substantial downstream accuracy gains.

With the commercialization of large language models (LLMs), weight-activation quantization has emerged to compress and accelerate LLMs, achieving high throughput while reducing inference costs. However, existing post-training quantization (PTQ) techniques for quantizing weights and activations of LLMs still suffer from non-negligible accuracy drops, especially on massive multitask language understanding. To address this issue, we propose Low-Rank Quantization (LRQ) - a simple yet effective post-training weight quantization method for LLMs that reconstructs the outputs of an intermediate Transformer block by leveraging low-rank weight-scaling matrices, replacing the conventional full weight-scaling matrices that entail as many learnable scales as their associated weights. Thanks to parameter sharing via low-rank structure, LRQ only needs to learn significantly fewer parameters while enabling the individual scaling of weights, thus boosting the generalization capability of quantized LLMs. We show the superiority of LRQ over prior LLM PTQ works under (i) 8-bit weight and per-tensor activation quantization, (ii) 4-bit weight and 8-bit per-token activation quantization, and (iii) low-bit weight-only quantization schemes. Our code is available at Software.