Retrieving Semantics for Fact-Checking: A Comparative Approach using CQ (Claim to Question) & AQ (Answer to Question)

Nicolò Urbani, Sandip Modha, Gabriella Pasi


Abstract
Fact-checking using evidences is the preferred way to tackle the issue of misinformation in the society. The democratization of information through social media has accelerated the spread of information, allowing misinformation to reach and influence a vast audience. The significant impact of these falsehoods on society and public opinion underscores the need for automated approaches to identify and combat this phenomenon.This paper is describes the participation of team IKR3-UNIMIB in AVeriTeC (Automated Verification of Textual Claims) 2024 shared task. We proposed a methods to retrieve evidence in the question and answer format and predict the veracity of a claim. As part of the AVeriTeC shared task, our method combines similarity-based ColBERT re-ranker with traditional keyword search using BM25. Additionally, a recent promising approach, Chain of RAG (CoRAG) is introduced to generate question and answer pairs (QAs) to evaluate performance on this specific dataset. We explore whether generating questions from claims or answers produces more effective QA pairs for veracity prediction. Additionally, we try to generate questions from the claim rather than from evidence (opposite the AVeriTeC dataset paper) to generate effective QA pairs for veracity prediction. Our method achieved an AVeriTeC Score of 0.18 (more than baseline) on the test dataset, demonstrating its potential in automated fact-checking.
Anthology ID:
2024.fever-1.3
Original:
2024.fever-1.3v1
Version 2:
2024.fever-1.3v2
Volume:
Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER)
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse, Zhenyun Deng, Mubashara Akhtar, Rami Aly, Zhijiang Guo, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal, James Thorne, Andreas Vlachos
Venue:
FEVER
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
37–46
Language:
URL:
https://aclanthology.org/2024.fever-1.3
DOI:
10.18653/v1/2024.fever-1.3
Bibkey:
Cite (ACL):
Nicolò Urbani, Sandip Modha, and Gabriella Pasi. 2024. Retrieving Semantics for Fact-Checking: A Comparative Approach using CQ (Claim to Question) & AQ (Answer to Question). In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pages 37–46, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Retrieving Semantics for Fact-Checking: A Comparative Approach using CQ (Claim to Question) & AQ (Answer to Question) (Urbani et al., FEVER 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.fever-1.3.pdf