Xingquan Zhang

Author directory

2026

Large Language Model (LLM)-based search agents have demonstrated superior performance in knowledge-intensive tasks through interleaved reasoning and search interactions. Process Reward Models (PRMs) can enhance search agent performance by guiding step selection during inference. However, existing PRMs for search agents face a critical bottleneck: assessment unreliability in their training data. This issue stems from an over-reliance on single-source supervision, forcing a compromise between global solvability and local validity. We propose CoPRM (Collaborative Process Reward Model), a process reward model trained on high-quality preference data generated through our Collaborative Calibration methodology. This approach synthesizes immediate quality assessments with outcome-informed evaluations through a dual-strategy mechanism: strictly enforcing consensus filtering to eliminate unreliable signals, and employing an outcome-informed gating mechanism to resolve heterogeneous cross-type trade-offs. Trained on 10k preference pairs, CoPRM provides universal test-time guidance, enabling diverse search agents to achieve consistent performance improvements across four multi-hop QA benchmarks, demonstrating substantial gains, robust cross-domain generalization, and superior efficiency.