Large Language Models (LLMs), with 7B+ parameters, remain difficult to deploy on edge devices due to memory latency constraints, despite increasing demand for personalized on-device agents with strong reasoning capabilities. Enabling such deployment, in current hardware specifications, requires aggressive compression while preserving downstream performance. Singular Value Decomposition (SVD)-based methods provide a promising reconstruction-aware compression paradigm, and recent trainable frameworks improve adaptive rank allocation under varying compression budgets. However, maintaining model quality under high compression remains challenging due to statistical mismatches between decomposed cores and compounded degradation from direct high-ratio compression.

We propose GRACE-SVD, a trainable SVD compression framework combining Guided Rank Allocation with a novel Core Equalization method that aligns decomposed core distributions while preserving reconstruction. We further introduce a progressive compression pipeline with intermediate equalization to mitigate performance collapse under aggressive compression. Across multiple model families and compression settings, our GRACE-SVD consistently outperforms existing SVD techniques, achieving stable compression even at 0.4 parameter ratio, with lower perplexity and stronger zero-shot performance under comparable or lower compression budgets.