LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Model

Zebin You, Xiaolu Zhang, Jun Zhou, Chongxuan Li, Ji-Rong Wen


Abstract
We present **LLaDA-o**, an effective and length-adaptive omni diffusion model for multimodal understanding and generation. LLaDA-o is built on a Mixture of Diffusion (MoD) framework that decouples discrete masked diffusion for text understanding and continuous diffusion for visual generation, while coupling them through a shared, simple, and efficient attention backbone that reduces redundant computation for fixed conditions. Building on MoD, we further introduce a data-centric length adaptation strategy that enables flexible-length decoding in multimodal settings without architectural changes. Extensive experiments show that LLaDA-o achieves strong performance on multimodal understanding and generation benchmarks, reaching 87.04 on DPG-Bench and 0.86 on GenEval. To support future work on omni-diffusion models, we release the model and training/evaluation code at https://github.com/ML-GSAI/LLaDA-o.
Anthology ID:
2026.findings-emnlp.221
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
4645–4664
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.221/
DOI:
Bibkey:
Cite (ACL):
Zebin You, Xiaolu Zhang, Jun Zhou, Chongxuan Li, and Ji-Rong Wen. 2026. LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Model. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 4645–4664, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Model (You et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.221.pdf