Diffusion language models (DLMs) are theoretically efficient but are limited by fixed-length decoding and poor compatibility with KV caches. Block diffusion mitigates these issues, yet still enforces a fixed block size and costly training. We introduce Next Sequence Prediction (NSP), which unifies next token and next block prediction, enabling the model to adaptively determine generation length at each step. NSP reduces to standard next token prediction when the length is one. Building on NSP, we propose the Sequential Diffusion Language Model (SDLM), which retrofits pretrained autoregressive language models (ALMs) at minimal cost. Specifically, SDLM performs diffusion inference within fixed-size mask blocks, but dynamically decodes contiguous subsequences based on model confidence, thereby preserving KV-cache compatibility and improving robustness to the varying uncertainty and semantics across the sequence. SDLM matches or surpasses strong autoregressive baselines with only 3.5M training samples, while achieving 2.0× 3.4× higher throughput than Qwen2.5 across three model scales (3B, 7B, 32B) with confidence-based decoding. Furthermore, when sampling with self-speculative decoding, SDLM attains up to 5.7× end-to-end speedup measured as wall-clock tokens per second (TPS). Notably, SDLM-32B achieves performance comparable to Qwen2.5-32B with up to 2.5× speedup, demonstrating the scalability of our approach. Project page and codes: https://github.com/OpenGVLab/SDLM

While Large Language Models (LLMs) have excelled in textual reasoning, they struggle with mathematical domains like geometry that intrinsically rely on visual aids. Existing approaches to Visual Chain-of-Thought (VCoT) are often limited by rigid external tools or fail to generate the high-fidelity, strategically-timed diagrams necessary for complex problem-solving. To bridge this gap, we introduce MathCanvas, a comprehensive framework designed to endow unified Large Multimodal Models (LMMs) with intrinsic VCoT capabilities for mathematics. Our approach consists of two phases. First, a Visual Manipulation stage pre-trains the model on a novel 15.2M-pair corpus, comprising 10M caption-to-diagram pairs (MathCanvas-Imagen) and 5.2M step-by-step editing trajectories (MathCanvas-Edit), to master diagram generation and editing. Second, a Strategic Visual-Aided Reasoning stage fine-tunes the model on MathCanvas-Instruct, a new 219K-example dataset of interleaved visual-textual reasoning paths, teaching it when and how to leverage visual aids. To facilitate rigorous evaluation, we introduce MathCanvas-Bench, a challenging benchmark with 3K problems that require models to produce interleaved visual-textual solutions. Our model, BAGEL-Canvas, trained under this framework, achieves an 86% relative improvement over strong LMM baselines on MathCanvas-Bench, demonstrating excellent generalization to other public math benchmarks. Our work provides a complete toolkit—framework, datasets, and benchmark—to unlock complex, human-like visual reasoning in LMMs.