Diagnosing Subword Boundary Conflicts in Diffusion Language Models through Sentence Simplification

Yaowen Xu, Ming Cai


Abstract
Sentence simplification is a source-grounded rewriting task in which outputs often keep parts of the source while editing adjacent spans. Diffusion language models (DLMs) are well suited to this setting because masked denoising can fill rewritten spans using bidirectional context. However, parallel filling can create subword-boundary errors when incompatible subword fragments are committed together. We define these errors as Subword Boundary Conflicts (SBCs): malformed output words, such as Australianways, formed from incompatible fragments at subword boundaries. To measure them, we introduce a surface-form taxonomy and a detection pipeline based on candidate extraction, LLM-as-a-Judge filtering, and human verification. We evaluate Dream and LLaDA variants on ASSET and MED-EASi, with autoregressive and task-specific simplification baselines. We find that SBCs occur across DLMs but vary by model. Under standard unmasking, LLaDA variants remain at or below 2.80%, whereas Dream-7B-Instruct reaches 6.32% on ASSET and 9.03% on MED-EASi. Beyond model choice, decoding parallelism is a key driver. In the Dream unmasking setting at 128 denoising steps, increasing parallelism from 1 to 4 raises SBC rates from near zero to 25.63% on ASSET and 34.67% on MED-EASi. Further analyses show that SBCs occur under light-to-moderate edits, are not limited to rare words, and can often be repaired by local masked span regeneration. These findings establish SBCs as a word-form reliability target for robust diffusion-based text generation.
Anthology ID:
2026.findings-emnlp.812
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
17196–17212
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.812/
DOI:
10.18653/2026.findings-emnlp.812
Bibkey:
Cite (ACL):
Yaowen Xu and Ming Cai. 2026. Diagnosing Subword Boundary Conflicts in Diffusion Language Models through Sentence Simplification. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 17196–17212, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Diagnosing Subword Boundary Conflicts in Diffusion Language Models through Sentence Simplification (Xu & Cai, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.812.pdf