SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs

Jiacheng Sang, Mengyuan Li, Sanxing Chen, Yukun Huang, Yu Feng, Bhuwan Dhingra


Abstract
LLM search agents are often evaluated on final-answer accuracy, overlooking the process. Analyzing a search strategy requires understanding how credible evidence is retrieved to address question constraints. This valuable information is buried in raw search trajectories that are long and difficult to parse. We introduce SearchAtlas, a framework that converts search trajectories into structured graphs whose edges represent how evidence is propagated across the reasoning trace, from the query that retrieves it to the final answer. Our automated parsing pipeline achieves a mean edge F₁ of 86.0% against human-annotated graphs and remains consistent across repeated runs. We analyze five search agents on three benchmarks, which reveal systematic differences in search scale and evidence aggregation across agents and benchmarks. SearchAtlas exposes fragmented answer support, question constraints that do not reach the answer, and unverified parametric knowledge entering the response. These process failures are strongly associated with incorrect answers, even more so than an LLM judge given either the raw trajectory or the ordered query list, suggesting that the constructed graphs provide useful interpretability. Moreover, an audit of cases in which process diagnostic scores disagree with final-answer correctness shows that they capture information not reducible to answer accuracy.
Anthology ID:
2026.findings-emnlp.2325
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
49907–49936
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2325/
DOI:
Bibkey:
Cite (ACL):
Jiacheng Sang, Mengyuan Li, Sanxing Chen, Yukun Huang, Yu Feng, and Bhuwan Dhingra. 2026. SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 49907–49936, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs (Sang et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2325.pdf