Shuangjie Fan

Author directory

2026

Many continual learning methods for LLMs mitigate forgetting by expanding task-specific modules, yet this leads to memory costs that grow with the number of tasks. We reveal that such expansion may be unnecessary: a single LoRA adapter possesses substantial per-task parameter redundancy, where retaining only 3–10% of LoRA-B entries suffices to recover over 95% of task accuracy. Moreover, the per-parameter update magnitudes follow a heavy-tailed distribution whose geometric knee point naturally separates critical parameters from dispensable ones. Building on this insight, we propose Continual Adaptation via Subspace Trimming (CAST), which adaptively allocates a compact task-specific subspace within a shared LoRA adapter using the knee point criterion, freezes previously assigned parameters to eliminate cross-task interference, and routes inputs to relevant subspaces via learned semantic vectors without requiring task identities. Experiments on LLaMA-3.1-8B and Qwen3-8B demonstrate that CAST matches per-task LoRA accuracy across 15 sequential tasks while maintaining a fixed adapter budget with over 24% idle capacity preserved.