Dimple: Discrete Diffusion Parallel Generation for Large Multimodal Modal

Runpeng Yu, Xinyin Ma, Xinchao Wang


Abstract
This paper introduces Dimple and Dimple+, two discrete diffusion multimodal large language models (dMLLMs). Dimple is derived from a discrete diffusion large language model (dLLM) that initially lacks multimodal understanding and acquires this capability through a hybrid training strategy that begins with autoregressive learning and subsequently transitions to discrete diffusion training. In contrast, Dimple+ is initialized from an autoregressive multimodal large language model and attains parallel decoding ability via purely discrete diffusion training. Both models achieve performance on par with their autoregressive counterparts, while Dimple+ further sets a new state of the art among existing dMLLMs. To improve inference efficiency, we propose Confident Decoding, a method that dynamically adapts the number of tokens generated at each iteration. Experimental results demonstrate decoding speedups of 2×–6× with only marginal performance loss. Additionally, we show that the Prefilling technique, previously developed for autoregressive models, can be effectively extended to dMLLMs with bidirectional attention, yielding nearly lossless acceleration of 1.7×–7×. Finally, we present the Structure Prior method, which enables fine-grained control over output format and reasoning structure, a capability that is challenging to achieve in autoregressive models.
Anthology ID:
2026.emnlp-main.824
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
19146–19170
Language:
URL:
https://aclanthology.org/2026.emnlp-main.824/
DOI:
Bibkey:
Cite (ACL):
Runpeng Yu, Xinyin Ma, and Xinchao Wang. 2026. Dimple: Discrete Diffusion Parallel Generation for Large Multimodal Modal. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 19146–19170, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Dimple: Discrete Diffusion Parallel Generation for Large Multimodal Modal (Yu et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.824.pdf