Guohong Li
Author directory2026
GNMR: Runtime Stability Control for Low-Precision Large Language Model Training
Boao Kong | Weichen Jia | Engao Zhang | Guohong Li | Yonghan Dong | Yao Wang | Yaoyuan Wang | Yunke Peng | Kun Yuan
Findings of the Association for Computational Linguistics: EMNLP 2026
Boao Kong | Weichen Jia | Engao Zhang | Guohong Li | Yonghan Dong | Yao Wang | Yaoyuan Wang | Yunke Peng | Kun Yuan
Findings of the Association for Computational Linguistics: EMNLP 2026
Training stability is a key bottleneck in low-precision language model training: efficient low-cost paths can still produce short-lived numerical risks at a small set of operators. We formulate this as runtime stability control and present **G**radient **N**orm-to-**M**ean **R**atio (**GNMR**), a lightweight controller that compares each recoverable unit’s current gradient norm with its historical mean. Together with 𝛥-GNMR for abrupt short-window increases, GNMR maps local risk signals to bounded recovery actions under a hard maxO budget and a short lock interval, without changing the numerical format, kernel, or backend recipe. Across activation-quantization stress, DeepSeek-style recipe-level training, and LLaMA-2 13B fine-tuning, GNMR preserves high-fidelity quality with sparse, budgeted recovery. These results support GNMR as a backend-agnostic controller to improve low-precision training stability while preserving low-cost execution.
2025
Towards Optimal Evaluation Efficiency for Large Language Models
Guohong Li | Deyi Xiong
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Guohong Li | Deyi Xiong
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Comprehensive evaluation of large language models (LLMs) typically requires large-scale benchmarks, which is costly in terms of both data annotation and computational resource needed for evaluation. To mitigate these challenges, we propose an efficient evaluation framework that selects a question subset based on pre-tested results, thereby reducing the costs. We formulate the subset selection problem as an optimization task, solved using optimal random sampling and simulated annealing algorithms. We compare our approach with prior clustering-based methods and assess their reliability in terms of score accuracy. Additionally, we perform semantic analysis and evaluate whether the selected subsets preserve the semantic information of the original benchmark using Wasserstein distance. Experimental results show that our method outperforms previous approaches in terms of reliability, as measured by L2 norm. Our study provides an optimized perspective for balancing evaluation efficiency and reliability in LLM assessments, while revealing the relationship between optimization methods and semantic retention.