Entropy mechanism emerges as a key organizing principle for understanding and improving Large Reasoning Models (LRMs). This survey examines how entropy shapes both their training and inference behavior. On the training side, we take a mathematical perspective: casting existing RL-based reasoning algorithms into a unified objective and using this formulation to derive an entropy-centric decomposition of methods, clarifying how different approaches adjust exploration–exploitation "knobs". On the inference side, based on the characteristic of LRM that trading increased inference tokens yields better performance, we summarize methods that leverage entropy to enhance inference performance or reduce uncertainty. Finally, we discuss open challenges and future directions of entropy-driven research for LRMs. Our repository is available on https://anonymous.4open.science/r/Awesome_LRM_with_Entropy-0503.

Large language models deployed as multi-step agents are increasingly trained via Direct Preference Optimization (DPO) on trajectory-level comparisons. Yet standard DPO weights every step uniformly, penalizing correct early actions in a losing trajectory as severely as the causal error itself. We propose Divergence-Weighted DPO(dwDPO), a drop-in replacement that detects the first divergence step between competing trajectories using implicit rewards already computed by DPO, then concentrates the loss around it via an asymmetric kernel with a closed-form scale factor that preserves the Bradley-Terry preference model. dwDPO recovers standard DPO as a special case and provably achieves higher gradient concentration at the critical step than both uniform and monotonic-discount baselines. Extensive experimental results across six deep-research benchmarks demonstrate the superiority of dwDPO over state-of-the-art open-source baselines of comparable scale at no additional inference cost.