Junwon Kim
Author directory2026
SurrogateKV: Representation-Preserving KV Cache Compression for Long-Context LLMs
Junwon Kim | Junghyun Ryu | Farid Talibli | Jungmin So | Youngjae Kim
Findings of the Association for Computational Linguistics: EMNLP 2026
Junwon Kim | Junghyun Ryu | Farid Talibli | Jungmin So | Youngjae Kim
Findings of the Association for Computational Linguistics: EMNLP 2026
KV cache compression is essential for long-context LLM inference. Most token-level KV compressors reduce cache length by retaining selected token KV entries and evicting the rest. Once evicted, those spans no longer receive attention, even though some may still carry information useful enough to preserve in compact form. We propose SurrogateKV, a training-free, runtime-only method that reframes KV compression as region-level representation allocation. Under the same KV-slot budget, SurrogateKV assigns historical regions to raw KV, surrogate KV, or drop. It does not train a compressor, modify model weights, or introduce a new attention operator. After prefill, SurrogateKV constructs lightweight surrogate entries as ordinary KV pairs, keeping their information available during standard attention decoding. This representation decision is orthogonal to layer- and head-wise budget allocation. We evaluate SurrogateKV-Snap, SurrogateKV-Dynamic, and SurrogateKV-Ada, obtained by applying SurrogateKV to SnapKV, DynamicKV, and Ada-KV, respectively. As representative results, SurrogateKV-Snap raises Needle-in-a-Haystack (NIAH) retrieval accuracy from 87.51 to 98.84 at 128 slots per layer. SurrogateKV-Ada raises the LongBench average score from 40.77 to 41.26 at 512 slots per layer while preserving 98.4% of full-cache performance with only 10.5% of the prefill KV cache. Code is available at https://github.com/kimjunewon/SurrogateKV.