Large Language Model (LLM) judges score responses by generating rating tokens, so their rewards carry uncertainty from the scoring mechanism itself, not only from genuine disagreement about quality. Sampling variance, prompt phrasing, and self-preference bias inject epistemic, generation-induced noise. Neighbour-Oriented Reward Anchoring (NORA) replaces generative scoring with retrieval-augmented regression: a small MLP, the Optimised Response Latent Inference (ORLI) judge, scores each query-response pair over a frozen embedding manifold anchored to human-rated neighbours. Because no language model generates the score, the reward is deterministic and its generation-induced variance is zero by construction, leaving a residual error (MAE=0.736 on the 1-5 scale) with no run-to-run component. This relocates the error rather than removing it: a per-tier breakdown shows the residual is dominated by systematic bias, the scorer compressing predictions toward the interior of the scale, rather than by aleatoric annotation noise. ORLI is still a learned reward, so NORA curbs over-optimisation with a retrieval-guided curriculum and a contrastive penalty against known-poor examples. Improvements are asymmetric and largest where quality is lowest: a 0.5B actor on Recipe4U raises 1-2 star responses by up to +0.77 ORLI points while leaving 5-star outputs essentially unchanged, the pattern holds on HelpSteer2, and a blinded human evaluation reproduces it. Ablations identify the curriculum as the primary contributor.