Sampling from the sequence-level power distribution elicits RL-level reasoning from base language models without any parameter updates, but the standard Metropolis–Hastings (MH), a Markov Chain Monte Carlo (MCMC) sampler, is both expensive and slow-mixing. We trace both to a structural mismatch: the power distribution mainly departs from the base distribution at a sparse, spatially clustered set of high-entropy decision points, yet MH proposes resampling positions uniformly along the prefix—wasting compute on near-degenerate conditionals while under-mixing precisely where modes diverge. We propose Entropy-Guided Power Sampling (EGPS), a training-free and verifier-free approximate sampler that re-derives its proposal from token-level entropy already in the forward pass. EGPS skips deterministic blocks, localizes each MCMC move to a high-entropy neighborhood, and applies Multiple-Try Metropolis at decision points—making sampling cost scale with entropy mass rather than sequence length. EGPS targets significant speedup with accuracy that matches or improves on MH power sampling: across three base models and three benchmarks it attains best or tied-best accuracy on 5 of 9 settings; on Qwen2.5-Math-7B it is best or tied-best on all three (MATH500 75.8%, HumanEval 62.2%, GPQA 42.4%), with wall-clock speedups over the MH baseline of up to 12.6× across these benchmarks. Our code is available at https://github.com/hongguo7/EGPS.