Consistency models (CMs) have shown promise in the efficient generation of both image and text. This raises the natural question of whether we can learn a unified CM for efficient multimodal generation (e.g., text-to-image) and understanding (e.g., image-to-text). Intuitively, such a model could be acquired by applying the consistency distillation (CD) to existing unified multimodal models. However, the key challenge is establishing a unified denoising perspective for both image and text generation, which is essential for establishing the consistency mapping. To tackle this, at the representation level, we advocate for discrete tokens for both modalities to best preserve language modeling capabilities. Critically, instead of defining the text denoising trajectory via recent discrete diffusion language modeling principles, we specify it using the parallel decoding trace of an autoregressive language model, benefiting from the latter’s superior performance in general text generation tasks. The denoising trajectory of image tokens adheres to standard discrete diffusion. We train our unified consistency models (UniCMs) on these combined multimodal trajectories simultaneously with a unified objective. We introduce a trajectory segmentation strategy to improve the training convergence. Empirically, in text-to-image generation, UniCMs outperform SD3 on GenEval and Image Reward, while requiring only approximately 1/8 of the sampling time. Meanwhile, in image-to-text generation, UniCMs surpass Show-o on the MMMU benchmark while being 1.5 × faster at long-sequence generating speed.

Diffusion large language models (dLLMs) accelerate language generation by predicting multiple masks in a single forward pass.However, existing dLLMs can suffer from unreliable predictions in early denoising stages under aggressive parallelism strategies, leading to errors that can propagate to later stages. To tackle this issue, we present Consistency Forcing (CForce) for dLLMs, a distillation method to force the mask predictions of early stages to align with those of later stages. CForce trains the model on pre-collected self-rollout trajectories, thereby improving training-inference alignment.We introduce Confidence Adaptive KL Divergence as a distillation objective to conjoin the merits of forward and reverse KL.We further provide a theoretical analysis for the consistency objective to explain why CForce can approximately minimize the prediction error of early stages.Critically, the same formulation applies to both mask-to-token decoding and edit-capable decoding; in the edit-capable case, later token-to-token refinements provide additional supervision for earlier masked-state predictions. Experiments on non-edit and edit-capable LLaDA models show improved speed-quality trade-offs, especially under high-parallelism decoding budgets.

Block Diffusion Language Models (BD-LMs) support KV caching and flexible-length generation, but their native Single-Block Diffusion (SingleBD) inference leaves inter-block parallelism unused. We study Multi-Block Diffusion (MultiBD), where a running-set of consecutive blocks is refined concurrently. Existing teacher-forced BD-LMs observe only one noisy block conditioned on a clean prefix, while diffusion forcing exposes a long noisy suffix whose state distribution still differs from practical MultiBD inference. We bridge this gap with Multi-Block Diffusion Language Models (MBD-LMs), obtained by post-training BD-LMs using Multi-block Teacher Forcing (MultiTF). MultiTF trains on bounded noise-groups conditioned on clean prefixes, with heterogeneous slot-wise noise patterns that better match inference. An optimized Block Buffer further preserves prefix-cache reuse and static input shapes, translating inter-block parallelism into wall-clock acceleration. On math and code benchmarks, MBD-LLaDA2-Mini increases average Tokens Per Forward pass (TPF) from 3.47 to 6.19 while maintaining comparable accuracy (80.0% versus 81.0%). Combined with DMax, it reaches 9.34 TPF with a 1.02-point accuracy drop. Project Page: https://sjtu-deng-lab.github.io/mbd-lms