Srinivas Harish

Author directory

2026

Running a tool agent n times and keeping the best run is worth 2.19–22.37 points on τ²-bench, a floor any scaffold spending n times the calls has to clear. Choosing among those runs, for no additional model calls, is worth anywhere from −23.68 to +12.28 depending on which rule does the choosing. We evaluate 9 selectors that read only the agent’s own trajectory across 28 model × domain cells and 10 403 released trials, and add an LLM judge that reads the trajectories but never a reward and never the evaluation criteria. Every arm is therefore reward-free: the nine rules are verifier-free as well, the judge is not, and neither sees a label. Against a mean best-of-K oracle band of 12.29 points, which is what a selector holding the reward would collect, the judge recovers 32.7% and the best free rule 21.0%, leaving 67.3% of that band unreachable without the label. One measurable property of the rollouts orders the task distributions, the no-mode rate, the share of tasks on which no two of the K rollouts emit the same write-action sequence, and what it predicts is when agreement is exploitable rather than when selection is possible. Where agreement-based rules are nearly inert the judge is worth +5.95 against +0.53; where the rollouts are undifferentiated it returns 1.19 against a band of 14.4 points; on a third domain it returns −0.64 and loses to a free heuristic costing one fewer call.
Search
Co-authors
    Venues
    Fix author