2026

Continual learning (CL) is essential for Large Language Models (LLMs) to sequentially adapt to evolving tasks. To mitigate catastrophic forgetting, recent advances implement low-rank adaptation with orthogonal projections (e.g., O-LoRA) to isolate task parameters. However, we reveal that such strict geometric constraints trigger an “Orthogonality Dilemma”: rigid parameter isolation impedes the transfer and accumulation of shared representations across semantically related tasks. In this work, we propose a new replay-free method, called  ̲Consolidation and  ̲Decoupling  ̲LoRA (CoDe-LoRA), for CL of LLMs. CoDe-LoRA disentangles the learning process into Consolidating\ Universal\ Knowledge and Decoupling\ Task-Specific\ Knowledge. To achieve this, CoDe-LoRA leverages an adaptive null-space projection mechanism and semantic routing to balance knowledge accumulation with task-specific adaptation. Experimental results on diverse CL benchmarks demonstrate that CoDe-LoRA consistently outperforms state-of-the-art methods. Our code is available at [https://github.com/Estrellajer/CoDe-LoRA](https://github.com/Estrellajer/CoDe-LoRA).