Large language models (LLMs) struggle with complex, long-horizon reasoning due to instability from the frozen-policy assumption. Current test-time scaling methods treat execution feedback merely as an external signal for filtering or rewriting trajectories, without internalizing it to improve reasoning strategies. Inspired by Popper’s “conjectures and refutations,” we posit that effective reasoning requires real-time policy adaptation through learning from failures. We introduce Policy of Thoughts (PoT), a framework that recasts reasoning as a within-instance online optimization process. PoT effectively generates diverse candidates via targeted exploration, then applies Group Relative Policy Optimization (GRPO) to update a transient LoRA adapter using execution feedback. This closed-loop design enables dynamic, instance-specific refinement of reasoning priors. Experiments demonstrate PoT’s substantial gains across multiple benchmarks. On LiveCodeBench, a 4B model achieves 49.71% accuracy, surpassing 12 test-time scaling baselines and commercial models (e.g., GPT-4o, DeepSeek-V3) despite being over 50 smaller.