While Parameter-Efficient Fine-Tuning (PEFT) reduces the cost of adapting Large Language Models (LLMs), current methods like LoRA suffer from catastrophic forgetting due to unregulated updates on high-importance parameters. To address this stability-plasticity dilemma, we introduce ANCHORS: Adaptive Normalization for Continual High-Order Retention via Sigmoid-Guided Magnitude Control, a novel framework that ensures uninterrupted knowledge retention during aggressive adaptation. ANCHORS integrates CABR Decomposition to preserve informative subspaces and S-MagNorm, a bio-inspired normalization mechanism that utilizes Sigmoid-guided magnitude control to dynamically dampen updates on critical weights. Extensive experiments on reasoning and QA benchmarks demonstrate that ANCHORS consistently outperforms LoRA, DoRA, and CUR-LoRA. It achieves performance gains of up to 4.05% in MCQ and 2.61% in QA tasks while retaining 61.12% of pre-trained knowledge—surpassing Experience Replay and EWC baselines in continuous learning scenarios.