Yash Sawant

Author directory

2026

Adaptive rank allocation for LoRA—allocating more parameters to important layers and fewer to unimportant ones—consistently improves efficiency under supervised fine-tuning (SFT). We test whether this success transfers to reinforcement learning, specifically Group Relative Policy Optimization (GRPO). Using gradient-magnitude profiling on Qwen 2.5 1.5B with GSM8K, we find that, in our setting, it does not: proportional rank allocation degrades accuracy by 4.5 points compared to uniform allocation (70.0% vs. 74.5%), despite using identical parameter budgets. We identify two mechanisms behind this failure. First, the gradient landscape under GRPO is fundamentally flatter than under SFT—the max-to-min layer importance ratio is only 2.17×, whereas the layer concentration reported by Shi et al. (2024) for SFT (top 30% of layers carrying 80% of the gradient signal) implies a max/min ratio well above 10×. All layers carry meaningful gradient signal; none are truly idle. Second, we observe a gradient amplification effect: non-uniform allocation widens the importance spread from 2.17× to 3.00×, creating a positive feedback loop where high-rank layers absorb more gradient while low-rank layers are progressively silenced. A random-allocation control yields the same amplification (r=0.972 correlation between assigned rank and resulting gradient share), indicating that rank causally determines gradient importance rather than the reverse. The negative result is single-seed and single-task; we present it as preliminary evidence that gradient importance does not predict capacity requirements under RL, and that naïve transfer of SFT-era rank allocation strategies to alignment training should be evaluated cautiously.
Search
Co-authors
    Venues
    Fix author