Jinseop Yeom

Author directory

2026

In this paper, we present MOB-KV, a Mixture-of-Basis KV cache compression method that makes adaptive low-rank key compression practical without runtime decomposition. We first analyze KV cache compression through attention output error and reveal a clear asymmetry between keys and values: pre-RoPE keys benefit from preserving dominant low-rank structure while quantizing low-energy residuals, whereas values are better compressed by direct quantization. This analysis motivates an asymmetric design that applies low-rank projection with residual quantization to keys and direct low-bit quantization to values. However, obtaining accurate input-adaptive low-rank bases with online SVD is prohibitively expensive for serving. MOB-KV addresses this by replacing online SVD with a reusable dictionary of offline-constructed key bases, transforming adaptive compression into lightweight basis selection and projection. This design preserves the accuracy benefit of local low-rank subspaces while avoiding the latency bottleneck of runtime decomposition. Across reasoning and long-context benchmarks, MOB-KV substantially reduces KV cache memory, preserves near full-precision accuracy, and establishes a stronger memory–accuracy–latency Pareto frontier than existing quantization-only, low-rank-only, and online SVD-based baselines.