Pruning is a critical technique for compressing Large Language Models (LLMs), yet its effectiveness is highly sensitive to the small calibration set used for parameter importance estimation. Current selection relies on simple heuristics like random sampling or entropy, causing large variation in post-pruning perplexity. In this work, we reveal the source of this instability at two levels: calibration samples are not equally important, and a distinct quality hierarchy, comprising Golden, Mediocre, and Detrimental samples, exists within any data pool. More importantly, we demonstrate that this hierarchy is not fixed: the utility of a sample is highly context-dependent, varying with the specific model and pruning algorithm, which renders static filtering approaches fundamentally insufficient. To address these, we introduce OASIS, the first end-to-end framework that directly optimizes calibration data selection with respect to the pruned model’s downstream performance. OASIS leverages a differentiable soft-mask proxy to propagate task-level gradients back to data importance weights. Comprehensive experiments across the Llama and Qwen families show that OASIS improves calibration robustness across the evaluated pruning settings, establishing a reliable standard for data-aware LLM compression.