Trung X. Pham

Author directory

2026

RL-trained search agents interleave retrieval with reasoning to answer knowledge-intensive questions, achieving strong end-to-end accuracy on multi-hop QA benchmarks. Yet end-to-end accuracy conflates two distinct abilities: searching for relevant documents and reading what they contain. Existing evaluations do not measure whether the agent reads the documents it retrieves. We design a controlled evaluation that force-injects gold-bearing documents into each agent’s retrieval pipeline, isolating reading from retrieval. Across five RL search agents on three multi-hop QA benchmarks, up to 46.2% of samples are answered incorrectly even when the gold answer is literally present in the retrieved documents—retrieval succeeds but reading fails. We term this phenomenon Retrieval Myopia (RM). The agent’s interaction format is itself a substantial cause: simply re-presenting the same documents to the same model in a plain QA format, with no weight updates, recovers up to 48.9% of RM cases. Guided by this diagnosis, we propose CWFT (Confidence-Weighted Format Transfer), a training-free pipeline that anchors the agent’s prediction against k confidence-weighted clean-format re-reads. CWFT consistently improves accuracy across all evaluated system–dataset configurations on three multi-hop QA benchmarks.
A calibrated Vision-Language Model (VLM) can repeatedly self-correct, say "Wait, I should recheck," arrive at the wrong answer, and still report high confidence. We find that this occurs because verbalized confidence is largely trajectory-independent in the VLMs and calibration methods we evaluate. We examine this through three complementary lenses: content variation, token masking, and the model’s own hesitation markers. We show that confidence is insufficiently sensitive to what the reasoning trajectory actually contains, and that calibration training can paradoxically worsen this disconnect. Since existing metrics like ECE and AUROC cannot detect this problem, we propose the Trajectory-Grounding Score (TGS) in two complementary forms: TGS-self, which compares confidence with and without access to the model’s own trajectory, and TGS-pair, which tests whether the model assigns higher confidence to correct trajectories than to flawed ones along the vision, reasoning, and answer axes. We propose TGS-Bench, a model-agnostic suite spanning 10 benchmarks with controlled good/bad trajectory pairs, and show that conventional calibration rankings diverge from trajectory-grounding rankings, exposing a blind spot in current evaluation practice.