Pengfei Guo
Author directory2026
SaToMe: Salience-Driven Token Merging for Omni-Capable Diffusion MLLMs
Sixun Liu | Pengfei Guo | Jinghao Zhang | Tingyu Chen | Xiaofeng Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Sixun Liu | Pengfei Guo | Jinghao Zhang | Tingyu Chen | Xiaofeng Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Omni-capable Diffusion multimodal large language models (Diffusion MLLMs) unify text-to-image (T2I) generation and image-to-text (I2T) understanding within one backbone, but at substantial cost on both fronts: visual tokens far outnumber surrounding text, and for T2I this per-step cost compounds across many denoising steps. Token merging has been studied separately in adjacent fields, yet training-free acceleration spanning T2I+I2T in one omni-capable Diffusion MLLM remains limited. We present SaToMe, a training-free token-merging method built around a local-deviation operator that we interpret as measuring local representational uniqueness. The denoising trajectory exposes two natural inputs to this operator: the state h and the update direction Δh. Their spreads behave very differently along the trajectory: state-salience spread is dynamic and identifies the active merging window, while update-salience masks are window-stable and serve as a single-step protection signal for cell-level structural content. T2I uses both signals; I2T, with no temporal axis, instantiates the same uniqueness criterion on the state alone, producing a single-pass additive merge. The same I2T form transfers directly to a separate I2T-only Diffusion MLLM with a different visual representation, supporting its applicability across these visual representations. SaToMe attains +20.14% T2I wall-clock speedup on LLaDA-o at 68% visual-token retention while largely preserving reported generation quality, +35.5% I2T wall-clock speedup on LLaDA-o, and +27.5% I2T speedup on LaViDa.