Pruning is a critical technique for compressing Large Language Models (LLMs), yet its effectiveness is highly sensitive to the small calibration set used for parameter importance estimation. Current selection relies on simple heuristics like random sampling or entropy, causing large variation in post-pruning perplexity. In this work, we reveal the source of this instability at two levels: calibration samples are not equally important, and a distinct quality hierarchy, comprising Golden, Mediocre, and Detrimental samples, exists within any data pool. More importantly, we demonstrate that this hierarchy is not fixed: the utility of a sample is highly context-dependent, varying with the specific model and pruning algorithm, which renders static filtering approaches fundamentally insufficient. To address these, we introduce OASIS, the first end-to-end framework that directly optimizes calibration data selection with respect to the pruned model’s downstream performance. OASIS leverages a differentiable soft-mask proxy to propagate task-level gradients back to data importance weights. Comprehensive experiments across the Llama and Qwen families show that OASIS improves calibration robustness across the evaluated pruning settings, establishing a reliable standard for data-aware LLM compression.

KV cache compression is commonly evaluated by final-answer accuracy, implicitly assuming that preserving the answer also preserves the reasoning that supports it. We test this assumption for large reasoning models and show that it can fail: under compression, answer-level cues and evidential support can remain differentially recoverable. We study this failure with a controlled fixed-trace replay protocol, which holds reasoning content fixed and isolates whether compression preserves usable information from an already available trace. We evaluate ten token-eviction KV compression methods and one quantization method on three models across mathematical reasoning, scientific QA, clinical calculation, and long-context retrieval. We measure final accuracy, answer-chain consistency, and perturbation faithfulness. Across tasks, token-eviction methods can preserve competitive final-answer accuracy while substantially degrading chain support or perturbation faithfulness. We call this the answer-evidence gap. A coverage-preserving quantization control is substantially less affected, suggesting that the failure is tied less to KV memory reduction itself than to losing access to parts of the reasoning trace.