Tianjian Feng
Author directory2026
Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space
Yangyi Shen | Tianjian Feng | Jiaqi Han | Wen Wang | Tianlang Chen | Chunhua Shen | Jure Leskovec | Stefano Ermon
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yangyi Shen | Tianjian Feng | Jiaqi Han | Wen Wang | Tianlang Chen | Chunhua Shen | Jure Leskovec | Stefano Ermon
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Diffusion Language Models (DLMs) offer order-agnostic generation that can explore many possible decoding trajectories. However, current decoding methods commit to a single trajectory, limiting exploration in trajectory space. We introduce Order-Token Search to explore this space through jointly searching over generation order and token values. Its core is a likelihood estimator that scores denoising actions, enabling stable pruning and efficient exploration of diverse trajectories. Across mathematical reasoning and coding benchmarks, Order-Token Search consistently outperforms baselines on GSM8K, MATH500, Countdown, and HumanEval (3.1%, 3.8%, 7.9%, and 6.8% absolute over backbone), reaching accuracy comparable to diffu-GRPO post-training. Our work establishes joint search as a key component for advancing decoding in DLMs.
Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models
Linhao Zhong | Linyu Wu | Bozhen Fang | Tianjian Feng | Chenchen Jing | Wen Wang | Jiaheng Zhang | Hao Chen | Chunhua Shen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Linhao Zhong | Linyu Wu | Bozhen Fang | Tianjian Feng | Chenchen Jing | Wen Wang | Jiaheng Zhang | Hao Chen | Chunhua Shen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Diffusion Language Models (DLMs) offer a promising alternative for language modeling by enabling parallel decoding through iterative refinement. However, most DLMs rely on hard binary masking and discrete token assignments, which hinder the revision of early decisions and underutilize intermediate probabilistic representations. In this paper, we propose EvoToken-DLM, a novel diffusion-based language modeling approach that replaces hard binary masks with evolving soft token distributions. EvoToken-DLM enables a progressive transition from masked states to discrete outputs, supporting revisable decoding. To effectively support this evolution, we introduce continuous trajectory supervision, which aligns training objectives with iterative probabilistic updates. Extensive experiments across multiple benchmarks show that EvoToken-DLM consistently achieves superior performance, outperforming strong diffusion-based and masked DLM baselines. Our code is available at https://github.com/aim-uofa/EvoTokenDLM.