Zifei Xu
Author directory2026
MF-QAT: Multi-Format Quantization-Aware Training for Elastic Inference
Zifei Xu | Sayeh Sharify | Hesham Mostafa
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Zifei Xu | Sayeh Sharify | Hesham Mostafa
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Quantization-aware training (QAT) is typically performed for a single target numeric format, while practical deployments often need to choose numerical precision at inference time based on hardware support or runtime constraints. We study multi-format QAT, which fine-tunes a pretrained model for multiple precisions within a microscaling format family. Across the evaluated language models, it achieves accuracy close to single-format QAT at most target precisions and generalizes to intermediate precisions unseen during fine-tuning, although the lowest-precision MXINT setting remains challenging. To make these models deployable from a single quantized checkpoint, we propose Slice-and-Scale, which converts a higher-precision MXINT or MXFP anchor to lower precisions by transforming both element values and shared block scales, without access to the original full-precision weights. Fine-tuning through this conversion produces an anchor checkpoint whose converted variants closely track standard multi-format QAT in language-modeling evaluations. Together, multi-format QAT and Slice-and-Scale enable precision selection from one stored checkpoint within each format family.