ACL-Verbatim: hallucination-free question answering for research

Gábor Recski, Szilveszter Tóth, Nadia Verdha, István Boros, Ádám Kovács


Abstract
Researchers need efficient and reliable methods for collecting high-quality information from trusted sources, but modern tools for AI-assisted research still suffer from the tendency of Large Language Models (LLMs) to produce factually inaccurate or nonsensical output, commonly referred to as hallucinations. We apply the extractive question answering system VerbatimRAG (CITATION) to research papers in the ACL Anthology[<https://aclanthology.org/>], directly mapping user queries to verbatim text spans in retrieved documents. We contribute a novel ground truth dataset for the task of mapping user queries to relevant text spans in research papers, and use it to train and evaluate a variety of extractive models. Human annotation is performed by NLP researchers and is based on synthetic user queries generated using a custom pipeline based on the ScIRGen methodology (CITATION), paired with chunks of research papers retrieved by VerbatimRAG. On this benchmark, a 150M-parameter ModernBERT token classifier trained on silver supervision from our pipeline achieves the best word-level F1 (53.6), ahead of the strongest evaluated LLM extractor (48.7).
Anthology ID:
2026.wasp-1.4
Volume:
Proceedings of the 4th Workshop on Artificial Intelligence for Scientific Publications
Month:
November
Year:
2026
Address:
Hengqin, Zhuhai, China
Editors:
Alberto Accomazzi, Tirthankar Ghosal, Felix Grezes, Kelly Lockhart, Atilla Kaan Alkan
Venues:
WASP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
23–33
Language:
URL:
https://aclanthology.org/2026.wasp-1.4/
DOI:
Bibkey:
Cite (ACL):
Gábor Recski, Szilveszter Tóth, Nadia Verdha, István Boros, and Ádám Kovács. 2026. ACL-Verbatim: hallucination-free question answering for research. In Proceedings of the 4th Workshop on Artificial Intelligence for Scientific Publications, pages 23–33, Hengqin, Zhuhai, China. Association for Computational Linguistics.
Cite (Informal):
ACL-Verbatim: hallucination-free question answering for research (Recski et al., WASP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.wasp-1.4.pdf