Nicolai Oswald

Author directory

2026

Diffusion language models (LMs) enable parallel decoding, but still lag behind autoregressive (AR) LMs in accuracy, and behind multi-token prediction (MTP) methods in efficiency. This raises three questions: should diffusion LMs compete with AR LMs or be harmonized with them; can diffusion provide a stronger acceleration mechanism than MTP; and does diffusion decoding have sufficient long-term potential to justify further exploration? We study these questions through Nemotron-Labs-Diffusion, a tri-mode LM that unifies three decoding modes—AR, diffusion, and self-speculation (diffusion drafts, AR verifies)—within a single model to sustain high throughput across all concurrency levels. Our study shows that (1) AR and diffusion objectives are mutually beneficial rather than competing, (2) diffusion-based self-speculation achieves higher acceptance lengths and better system vs. per-user throughput trade-offs than MTP baselines, and (3) diffusion decoding shows strong potential for parallel decoding under an optimal sampler in a speed-of-light analysis. Based on these insights, we develop the Nemotron-Labs-Diffusion family at 3B, 8B, and 14B scales. Our models outperform state-of-the-art open-source AR and diffusion LMs in both accuracy and speed, e.g., achieving 6x tokens per forward over Qwen3-8B with comparable benchmark accuracy, translating to 4x throughput on SPEED-Bench measured with SGLang on an NVIDIA GB200 GPU.