Yiwei Shi
Author directory2026
FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generation
Siyang He | Qiqi Wang | Xiaoran Liu | Hongnan Ma | Yiwei Shi | Yuerong Song | Ying Zhu | Tianyi Liang | Zengfeng Huang | Ziwei He | Xipeng Qiu
Findings of the Association for Computational Linguistics: EMNLP 2026
Siyang He | Qiqi Wang | Xiaoran Liu | Hongnan Ma | Yiwei Shi | Yuerong Song | Ying Zhu | Tianyi Liang | Zengfeng Huang | Ziwei He | Xipeng Qiu
Findings of the Association for Computational Linguistics: EMNLP 2026
Despite the non-autoregressive potential of diffusion language models (dLLMs), existing decoding strategies exhibit positional bias, failing to fully unlock the potential of arbitrary-order generation. In this work, we delve into the inherent spectral characteristics of dLLMs and present the first frequency-domain analysis of dLLMs, showing that low-frequency components in hidden states primarily encode global structural information and long-range dependencies, while high-frequency components capture local details. Based on this observation, we propose FourierSampler, which leverages a frequency-domain sliding window mechanism to dynamically guide the model to achieve "structure-to-detail" generation. FourierSampler outperforms other inference enhancement strategies on LLaDA and SDAR, achieving relative task-specific gains up to 20.4% on LLaDA-1.5 and 16.0% on LLaDA-8B-Instruct, as well as average gains up to 8.6% across tasks.