Robert Mullins
Author directoryAlso published as: Robert D. Mullins
2026
Scaling Laws for Mixed Quantization
Zeyu Cao | Yao Lai | Boyang Gu | Cheng Zhang | Pedro Gimenes | Jianqiao Lu | Xitong Gao | Jianyi Cheng | Robert D. Mullins | Yiren Zhao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zeyu Cao | Yao Lai | Boyang Gu | Cheng Zhang | Pedro Gimenes | Jianqiao Lu | Xitong Gao | Jianyi Cheng | Robert D. Mullins | Yiren Zhao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Post-training quantization (PTQ) of large language models (LLMs) reduces memory usage and accelerates inference while maintaining accuracy. However, it remains unclear how to select a quantization scheme that achieves target accuracy across different model scales. We introduce two parameters to characterize this: the quantization ratio Qr (proportion of parameters quantized to low precision) and the block size Qb (number of parameters sharing a scaling factor). Based on extensive experiments across multiple LLM families and scales, we propose a unified scaling law that predicts quantization-induced loss degradation as a function of Qr, Qb, and model size. Our scaling law reveals two practical implications: (1) larger models tolerate higher Qr, making mixed-precision increasingly effective at scale; (2) small Qb, commonly used in mixed precision, becomes unnecessary for large models, simplifying hardware design without sacrificing accuracy. These findings provide a principled recipe for quantizing LLMs in practical deployment.
Deep Kernel Fusion for Transformers
Zixi Zhang | Zhiwen Mo | Yiren Zhao | Robert D. Mullins
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Zixi Zhang | Zhiwen Mo | Yiren Zhao | Robert D. Mullins
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Agentic LLM inference with long contexts is increasingly limited by memory bandwidth rather than compute. In this setting, SwiGLU MLP blocks, whose large weights exceed cache capacity, become a major yet under-optimized bottleneck in the Transformer architecture. We propose DeepFusionKernel, a deeply fused kernel that cuts HBM traffic and boosts cache reuse, delivering up to 13.2% speedup on H100 and 9.7% on A100 over SGLang. Integrated with SGLang and paired with a kernel scheduler, DeepFusionKernel ensures consistent accelerations across generation lengths, while remaining adaptable to diverse models, inference configurations, and hardware platforms.
2023
Dynamic Stashing Quantization for Efficient Transformer Training
Guo Yang | Daniel Lo | Robert Mullins | Yiren Zhao
Findings of the Association for Computational Linguistics: EMNLP 2023
Guo Yang | Daniel Lo | Robert Mullins | Yiren Zhao
Findings of the Association for Computational Linguistics: EMNLP 2023
Large Language Models (LLMs) have demonstrated impressive performance on a range of Natural Language Processing (NLP) tasks. Unfortunately, the immense amount of computations and memory accesses required for LLM training makes them prohibitively expensive in terms of hardware cost, and thus challenging to deploy in use cases such as on-device learning. In this paper, motivated by the observation that LLM training is memory-bound, we propose a novel dynamic quantization strategy, termed Dynamic Stashing Quantization (DSQ), that puts a special focus on reducing the memory operations, but also enjoys the other benefits of low precision training, such as the reduced arithmetic cost. We conduct a thorough study on two translation tasks (trained-from-scratch) and three classification tasks (fine-tuning). DSQ reduces the amount of arithmetic operations by 20.95× and the number of DRAM operations by 2.55× on IWSLT17 compared to the standard 16-bit fixed-point, which is widely used in on-device learning.
Revisiting Automated Prompting: Are We Actually Doing Better?
Yulin Zhou | Yiren Zhao | Ilia Shumailov | Robert Mullins | Yarin Gal
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Yulin Zhou | Yiren Zhao | Ilia Shumailov | Robert Mullins | Yarin Gal
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Current literature demonstrates that Large Language Models (LLMs) are great few-shot learners, and prompting significantly increases their performance on a range of downstream tasks in a few-shot learning setting. An attempt to automate human-led prompting followed, with some progress achieved. In particular, subsequent work demonstrates that automation can outperform fine-tuning in certain K-shot learning scenarios. In this paper, we revisit techniques for automated prompting on six different downstream tasks and a larger range of K-shot learning settings. We find that automated prompting does not consistently outperform simple manual prompting. Our work suggests that, in addition to fine-tuning, manual prompting should be used as a baseline in this line of research.
Search