Other people with similar names: Lei Shi, Lei Shi, Lei Shi, Lei Shi, Lei Shi

Unverified author pages with similar names: Lei Shi


2026

Modern instruction-tuning mixtures are increasingly large, redundant, and heterogeneous, making full-data supervised fine-tuning costly and often unnecessary. We propose GRADFILTERING, an objective-agnostic, validation-free data selection method that scores examples using uncertainty-aware proxy training dynamics. GRADFILTERING trains a lightweight proxy model with a LoRA ensemble and ranks examples by a Gradient Signal-to-Noise Ratio (G-SNR), which combines early-to-late gradient drop with late-stage ensemble disagreement. Because selection requires neither target validation examples nor target-model gradients, the selected subsets can be reused across downstream targets. Across Alpaca and Tulu-3-52K, dense LLaMA-2-7B, LLaMA-3-8B, and Qwen3-8B targets, and a Qwen3-30B-A3B MoE target, GRADFILTERING-selected 5–15% subsets consistently outperform random selection and are competitive with the target-aware LESS-T reference while remaining validation-free. We further evaluate broad model capabilities on objective benchmarks and analyze proxy choice, gradient windows, ensemble size, human preferences, and selection cost, demonstrating the practical utility of early proxy dynamics for reusable, compact instruction-tuning data selection. Code is available at this GitHub repository.