TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning

Wonpyo Park, Seung-Won Hwang


Abstract
Long-context inference with large language models (LLMs) is constrained by the linear growth of the key–value (KV) cache to sequence length. While pruning offers mitigation, prevailing methods determine query-specific token importance that cannot be reused across unseen queries. In contrast, we introduce TaskPress, a framework for task-guided, query-agnostic KV cache eviction. Instead of optimizing the cache for a single query, TaskPress constructs a reusable memory representation conditioned on a high-level task guide. The guide functions as a "meta-query" during prefill to filter irrelevant tokens before downstream queries are issued. In addition, TaskPress leverages quantization scale factors as a zero-cost signal for detecting influential representation outliers, providing an efficient proxy for token importance. Experiments on conducted on various tasks with long context input demonstrate that TaskPress efficiently creates a compact, reusable cache across diverse queries.
Anthology ID:
2026.emnlp-main.793
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
18423–18434
Language:
URL:
https://aclanthology.org/2026.emnlp-main.793/
DOI:
Bibkey:
Cite (ACL):
Wonpyo Park and Seung-Won Hwang. 2026. TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 18423–18434, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning (Park & Hwang, EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.793.pdf