Yifan Hong

Author directory

2026

Continuous diffusion language models struggle with structured generation under hard, variable-length templates. We identify two failure modes: Boundary Attention Drift, in which head-region attention bleeds into padded tails under block-mode training, and Vocab Projection Drift, in which latent-to-token rounding over-projects to low-frequency tokens. The two modes dissociate cleanly across Chinese ci, English sonnets, and E2E NLG: the former is specific to block-mode training, whereas the latter appears in all three datasets regardless of padding regime, isolating padding regime and rounding rule as orthogonal sources of error. To address structural correctness, we adapt constrained decoding to diffusion rounding through a position-template primitive, the Vi rule, instantiated on continuous-diffusion logits as Length-Conditional Voronoi Rounding (LCVR). The same primitive applied to a small autoregressive baseline reaches identical zero-violation rates, showing that structural correctness is delivered by the template, not by the underlying model family. A contribution more readily realised with diffusion backbones is instead cross-domain backbone transfer: initialising a target-domain model from a structurally compatible source-domain backbone yields order-of-magnitude reductions in training steps, while a controlled negative-transfer arm fails to reproduce the speed-up on structural metrics; an attention-span probe localises the transferable component to the source backbone’s long-range attention span rather than to its padding regime or corpus size. We further report a calibrated wallclock analysis showing that the dominant cost of diffusion inference is recoverable with standard step-reduction techniques. Together, these results recast several previously scattered structured-generation failures of continuous diffusion as one consistent, transferable, and fixable phenomenon.