DIP: Dynamic In-Context Planner For Diffusion Language Models

Yang Li, Han Meng, Chenan Wang, Zhenyu Bi, Xuan Wang, Haipeng Chen


Abstract
Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples. Existing In-Context Learning (ICL) approaches largely inherit the practice of autoregressive language models (ARLMs), incorporating all examples into a fixed prompt. However, applying this rigid, static-prompt paradigm to DLMs incurs substantial computational overhead, as the model must evaluate the maximum context length at every step. We address this inefficiency with a key discovery: the block-wise KV-cache mechanism inherent to DLM inference enables the low-cost dynamic adjustment of the context. Following this intuition, our core idea is to start generation with a minimal prompt and progressively insert additional examples on the fly only when the generated tokens are of low confidence. Through rigorous empirical evaluations, we observe that average verified token confidence correlates strongly with generation accuracy, making it a reliable and computationally efficient signal of token quality. Formally, we propose Dynamic In-Context Planner (DIP), a context-optimization algorithm based on average verified confidence that dynamically ranks and inserts in-context examples during generation, rather than providing all examples up front. Experimental results on math and coding benchmarks with LLaDA-1.5 and LLaDA-8B-Instruct show that DIP achieves up to 1.59× and 1.36× speedups, respectively, while largely preserving the generation quality of the fixed-prompt baseline. Code: https://github.com/wmd3i/DIP
Anthology ID:
2026.findings-emnlp.713
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
14973–14991
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.713/
DOI:
Bibkey:
Cite (ACL):
Yang Li, Han Meng, Chenan Wang, Zhenyu Bi, Xuan Wang, and Haipeng Chen. 2026. DIP: Dynamic In-Context Planner For Diffusion Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 14973–14991, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
DIP: Dynamic In-Context Planner For Diffusion Language Models (Li et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.713.pdf