Dayuan Zhao

Author directory

2026

While autoregressive models (ARMs) dominate natural language processing, their strictly left-to-right decoding inherently limits global context utilization. Diffusion language models (DLMs) have recently emerged as a promising paradigm shift, theoretically offering bidirectional attention, parallel decoding, and the flexibility of any-order generation. However, the nature of this generative flexibility remains heavily under-explored. In this work, we systematically investigate the interplay between any-order flexibility, task performance, and inference efficiency. By quantifying deviations from autoregressive (AR) generation order using Normalized Edit Distance (NED) across reasoning, code, and commonsense tasks, we find that current DLMs rely heavily on near left-to-right generation pathways to achieve high performance. To weaken this coupling, we demonstrate that the generation block size acts as a key regulator of decoding order. Consequently, we propose a novel mitigation strategy: training the model with a large block size to grant it a vastly expanded exploration space, while performing inference with a comparatively smaller block size to minimize computational overhead. Ultimately, our methodology increases deviation from left-to-right generation while preserving strong task accuracy and inference efficiency.