Agentic reward models enhance large language model (LLM) evaluation by incorporating external tool-use capabilities. However, their training is heavily constrained by coarse-grained outcome supervision and the high cost of process-level annotations. To address these challenges, we propose SEAR (Self-judging agentic reward models), a novel training framework that enables agentic reward models to self-judge their own judging processes. SEAR introduces a dual-role adversarial co-evolution framework, where the model alternates between a judger and a self-judger within a single generation process: the judger is optimized to produce more rigorous reasoning trajectories that maximize self-judgment scores, while the self-judger learns to provide critical and discriminative evaluations. Moreover, SEAR further enables agentic RMs to self-evolve without relying on external human annotations. Extensive experiments demonstrate that SEAR consistently improves the reliability of agentic reward models across diverse benchmarks.

To facilitate the research on large language models (LLMs), this paper presents a comprehensive and unified library, LLMBox, to ease the development, use, and evaluation of LLMs. This library is featured with three main merits: (1) a unified data interface that supports the flexible implementation of various training strategies, (2) a comprehensive evaluation that covers extensive tasks, datasets, and models, and (3) more practical consideration, especially on user-friendliness and efficiency. With our library, users can easily reproduce existing methods, train new models, and conduct comprehensive performance comparisons. To rigorously test LLMBox, we conduct extensive experiments in a diverse coverage of evaluation settings, and experimental results demonstrate the effectiveness and efficiency of our library in supporting various implementations related to LLMs. The detailed introduction and usage guidance can be found at https://github.com/RUCAIBox/LLMBox.