DongNyeong Heo
Author directoryAlso published as: Dongnyeong Heo
2026
𝜆-Diffusion Language Models: Controllable Generalization of Continuous and Discrete Diffusion Dynamics
DongNyeong Heo | Hyunmin Song | Taehwan Kim
Findings of the Association for Computational Linguistics: EMNLP 2026
DongNyeong Heo | Hyunmin Song | Taehwan Kim
Findings of the Association for Computational Linguistics: EMNLP 2026
Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive language models. While discrete DLMs achieve strong performance across language generation tasks, continuous DLMs remain relatively underexplored despite their smooth latent-space dynamics. In this work, we present a unified embedding-space interpretation of continuous and discrete DLMs. Within this framework, we analyze continuous diffusion dynamics as a retention channel that preserves intermediate denoising information during sampling. Motivated by this perspective, we propose 𝜆-DLM, which smoothly interpolates between continuous and discrete diffusion dynamics through a controllable parameter 𝜆. By training with randomly sampled 𝜆, a single model generalizes across continuous, discrete, and intermediate diffusion dynamics, unifying diverse retention behaviors within one framework. Unlike prior methods that rely on fixed or self-confidence-guided diffusion dynamics, 𝜆-DLM enables controllable exploitation–exploration trade-offs without retraining. Experiments on LM1B and OWT unconditional generation benchmarks show that 𝜆-DLM consistently outperforms strong baselines. Moreover, varying 𝜆 enables semantically diverse generation without additional training. Compared with temperature sweeps, varying 𝜆 shows a more structured relationship between lexical diversity and sentence-embedding-based mode coverage.