Jinseop Yeom
Author directory2026
MOB-KV: Mixture-of-Basis for Key-Value Cache Compression
Jinseop Yeom | Taehyeon Kim | Eunhyeok Park
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jinseop Yeom | Taehyeon Kim | Eunhyeok Park
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
In this paper, we present MOB-KV, a Mixture-of-Basis KV cache compression method that makes adaptive low-rank key compression practical without runtime decomposition. We first analyze KV cache compression through attention output error and reveal a clear asymmetry between keys and values: pre-RoPE keys benefit from preserving dominant low-rank structure while quantizing low-energy residuals, whereas values are better compressed by direct quantization. This analysis motivates an asymmetric design that applies low-rank projection with residual quantization to keys and direct low-bit quantization to values. However, obtaining accurate input-adaptive low-rank bases with online SVD is prohibitively expensive for serving. MOB-KV addresses this by replacing online SVD with a reusable dictionary of offline-constructed key bases, transforming adaptive compression into lightweight basis selection and projection. This design preserves the accuracy benefit of local low-rank subspaces while avoiding the latency bottleneck of runtime decomposition. Across reasoning and long-context benchmarks, MOB-KV substantially reduces KV cache memory, preserves near full-precision accuracy, and establishes a stronger memory–accuracy–latency Pareto frontier than existing quantization-only, low-rank-only, and online SVD-based baselines.