Um, Actually: Timely and Calibrated Fact-Checking with DIVER

Senyi Zhang, Tasnim Kabir, Jordan Boyd-Graber


Abstract
Fact verification pipelines are often unrealistic: they assume full-input access, operate on isolated atomic claims, and overlook the central question of when a system should act under partial evidence. We introduce DIVER (Deliberative and Iterative fact VERification), an LLM-based framework for calibrated incremental fact-checking. DIVER incrementally extracts verifiable atomic claims from progressively revealed input, retrieves and refines evidence through iterative web search, and triggers a correction as soon as a claim is reliably refuted; if no such trigger occurs, it performs a passage-level revision over the full input to recover missed errors. Across standard fact-checking benchmarks, DIVER outperforms strong LLM-based verification pipelines, with especially large gains on retrieval-intensive settings. We further introduce UmActually, a real-world benchmark built from the Internet game Um, Actually, to evaluate not only whether a system finds an error, but whether it does so at the right time. On UmActually, DIVER reports the strongest end-to-end results while typically buzzing close to the earliest valid decision point, supporting its effectiveness for calibrated incremental fact-checking.
Anthology ID:
2026.findings-emnlp.855
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
18092–18106
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.855/
DOI:
Bibkey:
Cite (ACL):
Senyi Zhang, Tasnim Kabir, and Jordan Boyd-Graber. 2026. Um, Actually: Timely and Calibrated Fact-Checking with DIVER. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 18092–18106, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Um, Actually: Timely and Calibrated Fact-Checking with DIVER (Zhang et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.855.pdf