Many recursively self-improving systems improve by editing their codebases based on observed rollout behavior. However, recent methods largely treat self-improvement as offline search: variants are optimized and evaluated on fixed benchmarks. This misaligns with real-world deployment, where tasks arrive sequentially and may have different distributions. We study recursive self-improvement as online learning over non-stationary task streams. In this setting, reliable execution and fallback are particularly difficult with limited evidence: different branches of self-improvement see different tasks, so raw rewards conflate capability with task difficulty, leading to overly conservative or optimistic reversion. We propose Online Anchor-calibrated Self-Improvement System (OASIS) that runs an adaptive reference agent on incoming tasks, converts raw rewards into paired comparisons, and uses confidence-calibrated rules for branch continuation and fallback without evaluating every agent on all tasks. Across SWE-Bench Pro and DeepResearch Bench, OASIS consistently improves over both the initial agent and prior RSI baseline SSA, raising the SWE-Bench Pro solve rate from 51.6% to 56.9% using Qwen3.6-35B-A3B and DeepResearch Bench from 35.18 to 39.31 using Gemini 3 Flash. This enables OASIS to outperform larger open weights and proprietary systems, including Qwen3.5-397B-A17B, Minimax 2.7, Qwen3.6-Plus, and GPT-5.3-Codex, with sustained deep self-improvements rather than shallow cycles.