Huayi Jin
Author directory2026
Rethinking Diffusion Large Language Models via Time-aware Weighting and Block-oriented Refinements
Zheyong Xie | Shengrui Li | Li Zhong | Kaiyan Zhao | Fangcheng Shi | Jiakang Wang | Huayi Jin | Zhaokai Luo | Fei Zhao | Haifeng Liu | Jieying Ye | Chonggang Lu | Wenxuan Huang | Yue Liu | Zijie Meng | Shaosheng Cao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zheyong Xie | Shengrui Li | Li Zhong | Kaiyan Zhao | Fangcheng Shi | Jiakang Wang | Huayi Jin | Zhaokai Luo | Fei Zhao | Haifeng Liu | Jieying Ye | Chonggang Lu | Wenxuan Huang | Yue Liu | Zijie Meng | Shaosheng Cao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Diffusion Large Language Models (DLLMs) have recently emerged as a parallel alternative to autoregressive models, offering bidirectional context modeling and parallel decoding. However, developing highly performant DLLMs is hindered by lifecycle challenges, including suboptimal timestep weighting in pre-training, train-inference distribution gaps in post-training, and underutilized caching during inference. To address these obstacles, we introduce ReDLLM, a 30B-parameter Mixture-of-Experts DLLM. It incorporates a time-aware weighting strategy during continual pre-training that scales gradients according to denoising difficulty. Furthermore, we propose two block-oriented refinements: a terminal-block treatment to narrow the train-inference gap, and a block-aware inference pipeline featuring inline block prefill and block-level scheduling to maximize KV-cache reuse and parallelization. Extensive evaluations demonstrate that ReDLLM surpasses existing DLLMs at the same active-parameter scale, achieves generation quality competitive with 100B DLLMs, and attains the highest inference throughput among evaluated models.