Privacy-preserving adaptation of pretrained language models requires continual fine-tuning as private client distributions shift, while protecting individual samples under local differential privacy (LDP). In streaming differentially private federated learning, full-rank Low-Rank Adaptation (LoRA) adapters expose many trainable directions to DP noise, and later domains may overwrite directions that remain relevant to earlier domains. We propose Private Rank-Invariant Subspace Memory (PRISM), a fixed-rank LoRA framework that schedules the active plastic rank over the stream and applies shrinkage-gated memory writes to DP-protected aggregate updates. Server-side consolidation stores retained high-SNR directions in an orthogonal memory and remains pure post-processing of LDP updates. On MNLI genre and Sentiment3, PRISM stabilizes adaptation under strict privacy budgets while preserving a strong public initialization. At 𝜖=2 , it reaches 79.8% and 92.3% final average accuracy, compared with 52.3% and 85.0% for full-rank DP-FedSVD. Ablations indicate that rank scheduling accounts for most strict-budget recovery, and shrinkage-gated memory contributes a smaller retention gain and modestly improves over the strongest static low-rank control on MNLI.