Applying LLMs to query-document relevance estimation in industrial document retrieval faces two key challenges: the complexity and heterogeneity of Web content, and stringent latency and resource constraints. To address them, we propose an automated framework with two components: **R**eflection-**A**ware **C**ollaborative **A**gent (**RACA**) and **P**rogressive **O**ptimization **D**istillation **S**trategy (**PODS**). RACA uses collaborative reasoning and reflection to produce relevance labels and structured rationales. PODS stratifies training data by difficulty and combines supervised fine-tuning with Reasoning-Informed GRPO. Rationale supervision improves the student’s first-token relevance decision, while deployment stops after that token and avoids explicit rationale generation. Together, these mechanisms enable reasoning-informed first-token prediction that preserves the benefits of structured rationale supervision under stringent latency and resource constraints. Extensive offline experiments and a 10-day online A/B test in WeChat Search demonstrate the resulting accuracy–efficiency trade-off.