Large Language Models (LLMs), with 7B+ parameters, remain difficult to deploy on edge devices due to memory latency constraints, despite increasing demand for personalized on-device agents with strong reasoning capabilities. Enabling such deployment, in current hardware specifications, requires aggressive compression while preserving downstream performance. Singular Value Decomposition (SVD)-based methods provide a promising reconstruction-aware compression paradigm, and recent trainable frameworks improve adaptive rank allocation under varying compression budgets. However, maintaining model quality under high compression remains challenging due to statistical mismatches between decomposed cores and compounded degradation from direct high-ratio compression.

We propose GRACE-SVD, a trainable SVD compression framework combining Guided Rank Allocation with a novel Core Equalization method that aligns decomposed core distributions while preserving reconstruction. We further introduce a progressive compression pipeline with intermediate equalization to mitigate performance collapse under aggressive compression. Across multiple model families and compression settings, our GRACE-SVD consistently outperforms existing SVD techniques, achieving stable compression even at 0.4 parameter ratio, with lower perplexity and stronger zero-shot performance under comparable or lower compression budgets.

Large Language Models (LLMs) are pivotal in enabling intelligent experiences across various applications, from summarization to advanced content organization and retrieval functionalities. However, deploying LLMs for diverse tasks is fundamentally constrained by memory and compute limitations, making it impractical to fine-tune separate models for each task. Parameter-Efficient Fine-Tuning (PEFT) methods like Low-Rank Adaptation (LoRA) offer a scalable solution for multi-task LLM deployment. Despite its potential, LoRA faces challenges in selecting optimal ranks and layers for each task-model pair, often resulting in inefficiencies and unnecessary parameters. We introduce Norm Adaptive Localized (NormAL) LoRA, a novel variant that employs rank-norm regularization to dynamically determine the optimal rank for each weight matrix, ensuring adaptation is concentrated where it is most impactful. Our approach reduces adapter parameters by 37% while preserving full fine-tuning performance, making NormAL LoRA a transformative tool for enabling efficient, scalable, and space-constrained AI deployments across diverse industries and applications.