Large language models (LLMs) exhibit significant layer-wise redundancy, motivating non-uniform pruning for efficient compression. However, existing sparsity allocation methods rely on indirect statistical heuristics derived from weight magnitudes or activation distributions, without directly assessing how robustly each layer preserves semantic information under input variation. We propose PerturbPrune, a framework that allocates layer-wise sparsity by probing the intrinsic stability of hidden representations. PerturbPrune estimates the principal subspace of each layer’s activations via PCA and injects structured perturbations along these directions. Layer sensitivity is quantified through matrix difference entropy (DiME): stable layers are identified as encoding compact, essential features and thus preserved, while highly sensitive layers are deemed over-parameterized and pruned more aggressively. Extensive experiments across diverse LLM families (LLaMA, LLaMA-2, Qwen, OPT) demonstrate that PerturbPrune broadly matches or outperforms existing non-uniform methods. Compared to uniform pruning, it prevents catastrophic collapse, reducing perplexity by up to 99.7% and improving zero-shot accuracy by up to 22.7%. The method is also highly efficient, completing sparsity allocation on Qwen3.5-9B in 7 minutes—achieving 24× and 17.4× speedups over AlphaPruning and MRP. Finally, PerturbPrune generalizes to Mixture-of-Experts architectures, achieving up to 19.3% perplexity reduction on Qwen3.5-35B-A3B.

Reinforcement learning for open-ended text generation is constrained by the lack of verifiable rewards, necessitating reliance on judge models that require either annotated data or powerful closed-source models. Inspired by recent work on unsupervised reinforcement learning for mathematical reasoning using confidence-based endogenous rewards, we investigate whether this principle can be adapted to open-ended writing tasks. We find that directly applying confidence rewards leads to Triviality Bias: the policy collapses toward high-probability outputs, reducing diversity and meaningful content. We propose TCER (Triviality Corrected Endogenous Reward), which addresses this bias by rewarding the relative information gain between a specialist policy and a generalist reference policy, modulated by a probability-dependent correction mechanism. Across multiple writing benchmarks and model architectures, TCER achieves consistent improvements without external supervision. Furthermore, TCER also transfers effectively to mathematical reasoning, validating the generality of our approach across different generation tasks.

Although the effectiveness of Large Language Models as judges has been validated, their performance remains limited in open-ended tasks, particularly in story evaluation. Accurate story evaluation is crucial not only for assisting human quality judgment but also for providing reward signals to guide story generation. However, existing methods face a dilemma: prompt engineering for closed-source models suffers from poor adaptability, while fine-tuning approaches for open-source models lack the reasoning capabilities essential for story evaluation. To address this, we propose the Self-Evolving Pairwise Reasoning (EvolvR) framework. Grounded in pairwise comparison, the framework first self-synthesizes score-aligned Chain-of-Thought (CoT) data via a multi-persona strategy. To ensure data quality, these raw CoTs undergo a self-filtering process, utilizing multi-agents to guarantee their logical rigor and robustness. Finally, the evaluator trained on the refined data is deployed as a reward model to guide the story generation task. Experimental results demonstrate that our framework achieves state-of-the-art performance on three evaluation benchmarks including StoryER, HANNA and OpenMEVA. Furthermore, when served as a reward model, it enhances the quality of generated stories, thereby validating the superiority of our self-evolving approach.