AEScorer: An Agentic Evidence-Grounded Framework for Graded Factuality Verification

Hui Huang, Muyun Yang, Yuki Arase


Abstract
Despite the significant advancements of Large Language Models (LLMs), their factuality remains a critical challenge, creating a growing need for more nuanced factuality verification. Existing factuality verification methods do not capture graded judgments, even though factuality is better understood as a spectrum rather than a binary of right and wrong. To bridge this gap, we focus on graded factuality verification and propose AEScorer, an agentic evidence-grounded framework with two stages: agentic evidence acquisition and graded scoring. AEScorer first gathers and refines external evidence through agentic search, and then predicts a scalar factuality score to distinguish nuanced differences in factual correctness. We further construct GradedVeriBench, a benchmark for graded factuality verification spanning both general and multi-hop question answering. Experimental results on GradedVeriBench show that AEScorer substantially outperforms existing methods across both settings, demonstrating the value of coupling targeted evidence acquisition with graded scoring.
Anthology ID:
2026.findings-emnlp.331
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
6986–7004
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.331/
DOI:
Bibkey:
Cite (ACL):
Hui Huang, Muyun Yang, and Yuki Arase. 2026. AEScorer: An Agentic Evidence-Grounded Framework for Graded Factuality Verification. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 6986–7004, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
AEScorer: An Agentic Evidence-Grounded Framework for Graded Factuality Verification (Huang et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.331.pdf