First-order test-time optimization (TTO) updates continuous variables such as logits or latent states for a single input. Successful methods usually rely on external reward models or verifiers to supply a semantic gradient. We ask whether reward-free signals from model distributions or prompted self-verification can supply comparable update directions for reasoning tasks. In logit-space TTO, we compare each reward-free gradient with an operational reward-model reference direction, instantiated as the reward-model gradient on the same logits. Across Qwen2.5-7B-Instruct, Qwen3-14B, and Llama-3.1-8B-Instruct, confidence gradients remain nearly orthogonal to the reference, entropy-slope gradients have vanishing scale near greedy rollouts, and terminal answer certainty is sparse and reference-sensitive. Prompted-verifier gradients are more reference-dependent: in a tokenizer-matched Llama diagnostic, four variants attain moderate RM-relative alignment and measurable local RM-score gains. Nevertheless, in end-to-end TTO on Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct, the selected reward-free signals produce no wrong-to-correct flips on MATH-500, AIME-2024, AIME-2025, or GSM8K, while a separate reward-model-guided TTO reproduction provides an external positive control with measurable gains.