Improving Evidence Retrieval on Claim Verification Pipeline through Question Enrichment

Svetlana Churina, Anab Maulana Barik, Saisamarth Rajesh Phaye


Abstract
The AVeriTeC shared task introduces a new real-word claim verification dataset, where a system is tasked to verify a real-world claim based on the evidence found in the internet.In this paper, we proposed a claim verification pipeline called QueenVer which consists of 2 modules, Evidence Retrieval and Claim Verification.Our pipeline collects pairs of <Question, Answer> as the evidence. Recognizing the pivotal role of question quality in the evidence efficacy, we proposed question enrichment to enhance the retrieved evidence. Specifically, we adopt three different Question Generation (QG) technique, muti-hop, single-hop, and Fact-checker style. For the claim verification module, we integrate an ensemble of multiple state-of-the-art LLM to enhance its robustness.Experiments show that QueenVC achieves 0.41, 0.29, and 0.42 on Q, Q+A, and AVeriTeC scores.
Anthology ID:
2024.fever-1.6
Volume:
Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER)
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse, Zhenyun Deng, Mubashara Akhtar, Rami Aly, Zhijiang Guo, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal, James Thorne, Andreas Vlachos
Venue:
FEVER
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
64–70
Language:
URL:
https://aclanthology.org/2024.fever-1.6
DOI:
Bibkey:
Cite (ACL):
Svetlana Churina, Anab Maulana Barik, and Saisamarth Rajesh Phaye. 2024. Improving Evidence Retrieval on Claim Verification Pipeline through Question Enrichment. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pages 64–70, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Improving Evidence Retrieval on Claim Verification Pipeline through Question Enrichment (Churina et al., FEVER 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.fever-1.6.pdf