Weichen Jia
Author directory2026
GNMR: Runtime Stability Control for Low-Precision Large Language Model Training
Boao Kong | Weichen Jia | Engao Zhang | Guohong Li | Yonghan Dong | Yao Wang | Yaoyuan Wang | Yunke Peng | Kun Yuan
Findings of the Association for Computational Linguistics: EMNLP 2026
Boao Kong | Weichen Jia | Engao Zhang | Guohong Li | Yonghan Dong | Yao Wang | Yaoyuan Wang | Yunke Peng | Kun Yuan
Findings of the Association for Computational Linguistics: EMNLP 2026
Training stability is a key bottleneck in low-precision language model training: efficient low-cost paths can still produce short-lived numerical risks at a small set of operators. We formulate this as runtime stability control and present **G**radient **N**orm-to-**M**ean **R**atio (**GNMR**), a lightweight controller that compares each recoverable unit’s current gradient norm with its historical mean. Together with 𝛥-GNMR for abrupt short-window increases, GNMR maps local risk signals to bounded recovery actions under a hard maxO budget and a short lock interval, without changing the numerical format, kernel, or backend recipe. Across activation-quantization stress, DeepSeek-style recipe-level training, and LLaMA-2 13B fine-tuning, GNMR preserves high-fidelity quality with sparse, budgeted recovery. These results support GNMR as a backend-agnostic controller to improve low-precision training stability while preserving low-cost execution.