Despite the remarkable progress of reinforcement learning for large language models, effective exploration remains a persistent challenge, as policies may collapse into recurrent failure modes and repeatedly generate similar erroneous behaviors across rollouts. While classical entropy regularization promotes randomness under the current policy, it provides limited mechanisms for recognizing and suppressing such recurrent failure patterns over time. To address this limitation, we propose MEDS, a Memory-Enhanced Dynamic reward Shaping framework that augments the reward signal with informative cues from historical behaviors. MEDS turns intermediate model representations into a lightweight behavioral memory that captures reasoning patterns from past rollouts, enabling dynamic reward shaping that steers the policy away from recurrent failure modes and toward alternative solution paths. Across five datasets and three base models, MEDS consistently improves average performance over existing baselines, achieving gains of up to 4.13 Pass@1 points and 4.37 Pass@128 points. LLM-based annotations and diversity metrics further confirm that MEDS promotes more diverse sampling behaviors. Our code is available at https://github.com/Linxi000/MEDS.