Retrieval-Augmented Generation (RAG) mitigates hallucinations by grounding generation in retrieved evidence, yet failures persist when models underutilize the retrieved context. While prior work attributes this to conflicts between external context and internal parametric knowledge encoded in Feed-Forward Networks (FFNs), our empirical analyses reveal that FFN-derived signals are unstable and insufficient for reliable diagnosis across diverse contexts. We therefore propose the Attention Escape Score (AES), a training-free diagnostic framework that provides a mechanistic lens on RAG hallucinations via dynamic attention routing failures. We models the failures as weakened Semantic Anchoring, increased Contextual Distraction, and heightened Generation Inertia, moving beyond static component-level analyses. Extensive experiments show that AES significantly outperforms state-of-the-art baselines with robust cross-model generalization. Furthermore, we introduce Pruning Escape Routes (PER), a lightweight inference-time intervention that suppresses hallucinations by attenuating identified escape pathways, achieving improved faithfulness with negligible degradation to complex reasoning and general capabilities.

Large language models (LLMs) struggle with complex, long-horizon reasoning due to instability from the frozen-policy assumption. Current test-time scaling methods treat execution feedback merely as an external signal for filtering or rewriting trajectories, without internalizing it to improve reasoning strategies. Inspired by Popper’s “conjectures and refutations,” we posit that effective reasoning requires real-time policy adaptation through learning from failures. We introduce Policy of Thoughts (PoT), a framework that recasts reasoning as a within-instance online optimization process. PoT effectively generates diverse candidates via targeted exploration, then applies Group Relative Policy Optimization (GRPO) to update a transient LoRA adapter using execution feedback. This closed-loop design enables dynamic, instance-specific refinement of reasoning priors. Experiments demonstrate PoT’s substantial gains across multiple benchmarks. On LiveCodeBench, a 4B model achieves 49.71% accuracy, surpassing 12 test-time scaling baselines and commercial models (e.g., GPT-4o, DeepSeek-V3) despite being over 50 smaller.