Diffusion language models (DLMs) are theoretically efficient but are limited by fixed-length decoding and poor compatibility with KV caches. Block diffusion mitigates these issues, yet still enforces a fixed block size and costly training. We introduce Next Sequence Prediction (NSP), which unifies next token and next block prediction, enabling the model to adaptively determine generation length at each step. NSP reduces to standard next token prediction when the length is one. Building on NSP, we propose the Sequential Diffusion Language Model (SDLM), which retrofits pretrained autoregressive language models (ALMs) at minimal cost. Specifically, SDLM performs diffusion inference within fixed-size mask blocks, but dynamically decodes contiguous subsequences based on model confidence, thereby preserving KV-cache compatibility and improving robustness to the varying uncertainty and semantics across the sequence. SDLM matches or surpasses strong autoregressive baselines with only 3.5M training samples, while achieving 2.0× 3.4× higher throughput than Qwen2.5 across three model scales (3B, 7B, 32B) with confidence-based decoding. Furthermore, when sampling with self-speculative decoding, SDLM attains up to 5.7× end-to-end speedup measured as wall-clock tokens per second (TPS). Notably, SDLM-32B achieves performance comparable to Qwen2.5-32B with up to 2.5× speedup, demonstrating the scalability of our approach. Project page and codes: https://github.com/OpenGVLab/SDLM

We present AnyCap, a plug-and-play framework that brings instruction alignment to omni-modal captioning. Captions offer a unified language interface for multimodal learning, and users increasingly expect instruction-driven control over their content and style. Current caption models lack explicit instruction supervision and are weak at instruction following, while directly tuning them can degrade general language ability. Achieving instruction alignment in an omni-modal setting is harder still, as each modality calls for separate models and custom designs. To address these challenges, AnyCap leverages a multimodal revision-based refinement paradigm that refines uncontrolled captions from existing models to instruction-aligned ones, without re-training base models. By processing multi-modality features in a unified framework, it enables one model to serve images, videos, and audio. To address the lack of instruction-based data, we construct AnyCapData, a large-scale, high-quality corpus spanning three modalities with 28 well-designed instruction types, each with multiple instruction samples. For evaluation, we address the limitations of current metrics for instruction-oriented captioning by designing AnyCapEval. Its key insight is to decouple evaluation into content and style for fine-grained assessment. Extensive experiments show that on AnyCapEval and diverse public benchmarks, AnyCap consistently improves both caption quality and instruction adherence for both open-source and API-based models. Notably, AnyCap-8B boosts GPT-4o’s content scores by 46% and style scores by 12%. Our code and models will be made publicly available.