While Parameter-Efficient Fine-Tuning (PEFT) reduces the cost of adapting Large Language Models (LLMs), current methods like LoRA suffer from catastrophic forgetting due to unregulated updates on high-importance parameters. To address this stability-plasticity dilemma, we introduce ANCHORS: Adaptive Normalization for Continual High-Order Retention via Sigmoid-Guided Magnitude Control, a novel framework that ensures uninterrupted knowledge retention during aggressive adaptation. ANCHORS integrates CABR Decomposition to preserve informative subspaces and S-MagNorm, a bio-inspired normalization mechanism that utilizes Sigmoid-guided magnitude control to dynamically dampen updates on critical weights. Extensive experiments on reasoning and QA benchmarks demonstrate that ANCHORS consistently outperforms LoRA, DoRA, and CUR-LoRA. It achieves performance gains of up to 4.05% in MCQ and 2.61% in QA tasks while retaining 61.12% of pre-trained knowledge—surpassing Experience Replay and EWC baselines in continuous learning scenarios.

Current RLHF methods such as PPO and DPO typically reduce human preferences to binary labels, which are costly to obtain and too coarse to reflect individual variation. We observe that expressions of satisfaction and dissatisfaction follow stable linguistic patterns across users, indicating that more informative supervisory signals can be extracted from free-form feedback. Building on this insight, we introduce Adaptive Reward-Following (ARF), which converts natural feedback into continuous preference trajectories and optimizes them using the novel TraceBias algorithm. Across diverse LLMs and preference domains, ARF consistently outperforms PPO and DPO, improving alignment by up to 7.6%. Our results demonstrate that continuous reward modeling provides a scalable path toward personalized and theoretically grounded RLHF.