KV cache compression is essential for scaling large language models to long-context inference. However, existing methods excel at explicit retrieval tasks but often degrade on tasks that require multi-step reasoning over dispersed evidence. We present **ClueKV**, a training-free KV cache compression framework that better preserves the reasoning-supportive clues under tight cache budgets. The core design of ClueKV is a self-guided, adaptive headwise token reweighting mechanism. Specifically, ClueKV first uses contrastive probability shifts between the query-conditioned context and a context-free reference to construct an unsupervised clue signal, which highlights context tokens supportive of downstream reasoning under the current input. This signal then serves as self-generated guidance for compression. Based on this signal, ClueKV dynamically identifies, for each input instance, the attention heads that are more aligned with reasoning-relevant context regions, and reweights token importance head by head before KV eviction. Across LongReason, LongBench, and RULER, ClueKV consistently outperforms strong KƒV compression baselines, with the largest gains on complex reasoning tasks. On LongReason, it retains 99.21% of full-KV accuracy at a 50% cache budget, and remains more stable than prior methods as context length increases to 64k tokens. ClueKV also introduces only marginal prefill overhead, making it practical for long-context deployment.

Despite the remarkable progress of Large Language Models (LLMs), the escalating memory footprint of the Key-Value (KV) cache remains a critical bottleneck for efficient inference. While dimensionality reduction offers a promising compression avenue, existing approaches typically either necessitate prohibitively expensive pre-training from scratch or suffer from severe performance deterioration under high compression regimes. In this work, we propose DynaKV, a novel post-training framework for low-rank KV cache compression. To the best of our knowledge, DynaKV is the first low-rank method to dynamically allocate compression rates to individual tokens according to their semantic meaning, which allows it to achieve better fidelity at aggressive compression ratios. Extensive experiments demonstrate that our method consistently outperforms existing state-of-the-art compression techniques, achieving significant memory reduction while maintaining competitive generation quality. Furthermore, our approach is orthogonal to sequence-level pruning methods. When integrated with SnapKV, DynaKV retains only 6% of the KV cache while maintaining 94% of the baseline performance on the LongBench benchmark.

Privacy-sensitive users require deploying large language models (LLMs) within their own infrastructure (on-premises) to safeguard private data and enable customization. However, vulnerabilities in local environments can lead to unauthorized access and potential model theft. To address this, prior research on small models has explored securing only the output layer within hardware-secured devices to balance model confidentiality and customization. Yet this approach fails to protect LLMs effectively. In this paper, we discover that (1) query-based distillation attacks targeting the secured top layer can produce a functionally equivalent replica of the victim model; (2) securing the same number of layers, bottom layers before a transition layer provide stronger protection against distillation attacks than top layers, with comparable effects on customization performance; and (3) the number of secured layers creates a trade-off between protection and customization flexibility. Based on these insights, we propose SOLID, a novel deployment framework that secures a few bottom layers in a secure environment and introduces an efficient metric to optimize the trade-off by determining the ideal number of hidden layers. Extensive experiments on five models (1.3B to 70B parameters) demonstrate that SOLID outperforms baselines, achieving a better balance between protection and downstream customization.