Memory bottlenecks severely constrain the deployment of large language models (LLMs) in long-context scenarios. While existing global pruning methods reduce memory usage during the decoding phase, they largely ignore out-of-memory errors in the prefill stage before generation begins. To address this, we present Task-Aware Chunked KV Cache Eviction (TAKE), a training-free framework for memory-efficient long-context prefill. TAKE refines the prefill process by partitioning long sequences into chunks and performing incremental KV eviction, thereby avoiding the computational overhead of full-sequence processing. Crucially, to mitigate the semantic discontinuity introduced by chunk-wise processing, we introduce accumulated task-aware probe tokens and a delayed eviction strategy to reduce information loss. Extensive evaluations show that TAKE achieves the highest efficiency, slashing peak memory usage to 59.53% of the baseline while accelerating Time-To-First-Token (TTFT) by over 60% under 128k-token inference. Our code is available at [Source Code of TAKE](https://github.com/ObitoJN/TAKE).

On-policy distillation (OPD) trains a student on its own trajectories with token-level teacher feedback and often outperforms off-policy distillation and reinforcement learning with verifiable rewards. However, we find that its standard advantage-weighted policy gradient suffers from three structural weaknesses, including high-variance updates, vanishing gradients in zero-advantage regions, and exploration bottlenecks for models with limited capability. We therefore propose asymmetric on-policy distillation (AOPD), which replaces ineffective policy gradient with localized divergence minimization in non-positive advantage regions while preserving positive reinforcement learning. Experiments on mathematical reasoning benchmarks show that AOPD consistently outperforms standard OPD, with average gains of 4.09 / 8.34 under strong / weak initialization, respectively. AOPD also maintains higher policy entropy during training and better capability retention during sequential tool-use adaptation. Our source code is available at [repository](https://github.com/stargazer319/AOPD).