Huang Yinjun

Author directory

2026

Long-context inference in large language models is limited by the linear memory growth of the key-value (KV) cache. Existing eviction methods typically retain tokens with high accumulated attention or remove semantic redundancy, but may discard low-frequency tokens that connect distant dependencies. We propose CentralKV, a training-free framework that models attention as a dynamic sparse causal graph. Its Stream-Influence algorithm updates token centrality online and preserves structurally important bridge tokens during KV compression. The attention graph is constructed directly from the attention operation without separate similarity computation. Experiments on LongBench show that CentralKV improves over H2O, SnapKV, and GraphKV under strict physical memory constraints, particularly on reasoning-intensive tasks, with minimal latency overhead.