Jinyoung Choi
Author directory2026
Signals from the Future: Future-aware Semi-Autoregressive Decoding for Diffusion Language Models
Yangryeol Park | Jeonhwi Kang | Kunhui Lee | Hyeonju Kim | Jinyoung Choi | Seung-Hoon Na
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yangryeol Park | Jeonhwi Kang | Kunhui Lee | Hyeonju Kim | Jinyoung Choi | Seung-Hoon Na
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Standard semi-autoregressive decoding for full-attention masked diffusion language models (MDLMs) computes predictions for future blocks but discards them until those blocks become active. When introduced as soft-future context, these predictions reveal a stability contrast: hypotheses later revised during decoding exhibit larger distributional shifts than those eventually retained. We introduce FUSA (Future-aware Semi-Autoregressive Decoding), a training-free decoder that exploits this contrast by alternating the next block between masked and soft representations and committing active-block tokens that remain stable across the two views, without additional model forward calls. On LLaDA-8B-Instruct, FUSA improves scores across reasoning and code-generation benchmarks while delivering 2.10– 3.94× wall-clock speedups over Top-1 decoding, with gains of up to 5.1 points. Additional results show that FUSA transfers to other MDLMs and remains effective under KV-cached inference and in long-form instruction following. Our code is available at https://github.com/yangryeol/FUSA.