Reinforcement learning (RL) for retrieval-augmented language models enables policies to interleave generated reasoning with retrieved observations. However, in Search-R1-style PPO, training may become unstable after the model has already reached strong validation performance. This paper studies late-stage collapse as a training-stability problem in search-based RL. Four metrics are proposed to quantify validation drawdown (VDD), advantage-tail expansion (ATE), actor update stress (AUS), and observation-to-actor contamination in generalized advantage estimation (GAE), denoted as OAGC. The analysis further shows how observation tokens can influence actor updates through value targets, value-model learning, and GAE. To address this issue, Observation-aware Stabilization (OAS) combines observation residual decay, old-value anchoring, and boundary smoothing. On three benchmark datasets (NQ, TriviaQA, and HotpotQA), OAS improves Search-R1 PPO under multiple retrieval budgets. In the main Top-3 setting, our method improves the exact match (EM) measure from 31.8/49.6/18.7 to 37.6/56.8/23.9 and reduces OAGC from 0.331 to 0.088, with lower search-budget saturation and no increase in the no-search rate.