Exploring Retrieval Augmented Generation For Real-world Claim Verification

Adjali Omar


Abstract
Automated Fact-Checking (AFC) has recently gained considerable attention to address the increasing misinformation spreading in the web and social media. The recently introduced AVeriTeC dataset alleviates some limitations of existing AFC benchmarks. In this paper, we propose to explore Retrieval Augmented Generation (RAG) and describe the system (UPS participant) we implemented to solve the AVeriTeC shared task.Our end-to-end system integrates retrieval and generation in a joint training setup to enhance evidence retrieval and question generation. Our system operates as follows: First, we conduct dense retrieval of evidence by encoding candidate evidence sentences from the provided knowledge store documents. Next, we perform a secondary retrieval of question-answer pairs from the training set, encoding these into dense vectors to support question generation with relevant in-context examples. During training, the question generator is optimized to generate questions based on retrieved or gold evidence. In preliminary automatic evaluation, our system achieved respectively 0.198 and 0.210 AVeriTeC scores on the dev and test sets.
Anthology ID:
2024.fever-1.13
Volume:
Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER)
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse, Zhenyun Deng, Mubashara Akhtar, Rami Aly, Zhijiang Guo, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal, James Thorne, Andreas Vlachos
Venue:
FEVER
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
113–117
Language:
URL:
https://aclanthology.org/2024.fever-1.13
DOI:
Bibkey:
Cite (ACL):
Adjali Omar. 2024. Exploring Retrieval Augmented Generation For Real-world Claim Verification. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pages 113–117, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Exploring Retrieval Augmented Generation For Real-world Claim Verification (Omar, FEVER 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.fever-1.13.pdf