High-quality preference data collection remains a bottleneck for RLHF reward model training: collecting multiple independent human annotations per sample is costly and hard to scale across domains and languages, while fully automated AI feedback (RLAIF) risks quality degradation. We present a production-grade hybrid human-AI annotation framework that introduces automated model-as-annotator pre-annotations into a quality-control (QC)-ratified workflow, contributing model votes only when automation is sufficiently reliable and leaving final adjudication to human QC. The pipeline (1) verifies factual claims with a web-search-grounded fact checker under a cleanroom constraint that keeps proprietary tokens out of training data, (2) elicits structured judgments from an ensemble of open-weights LLM judges, and (3) applies a two-call swap-consistency gate to mitigate position bias, with a data flywheel surfacing disagreements between QC and model outputs to drive improvement. In production, the framework reduces initial-labeler hours by ∼37%; across 11,092 production comparisons it reaches 81.6% agreement with QC (𝜅=0.63) and 82% to 88% on the public JudgeLM benchmark, with no QC degradation measured under an independent-model proxy. The resulting data supported a production reward model and aligned LLM.