FactGenius: Combining Zero-Shot Prompting and Fuzzy Relation Mining to Improve Fact Verification with Knowledge Graphs

Sushant Gautam, Roxana Pop


Abstract
Fact-checking is a crucial natural language processing (NLP) task that verifies the truthfulness of claims by considering reliable evidence. Traditional methods are labour- intensive, and most automatic approaches focus on using documents as evidence. In this paper, we focus on the relatively understudied fact-checking with Knowledge Graph data as evidence and experiment on the recently introduced FactKG benchmark. We present FactGenius, a novel method that enhances fact- checking by combining zero-shot prompting of large language models (LLMs) with fuzzy text matching on knowledge graphs (KGs). Our method employs LLMs for filtering relevant connections from the graph and validates these connections via distance-based matching. The evaluation of FactGenius on an existing benchmark demonstrates its effectiveness, as we show it significantly outperforms state-of- the-art methods. The code and materials are available at https://github.com/SushantGautam/FactGenius.
Anthology ID:
2024.fever-1.30
Volume:
Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER)
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse, Zhenyun Deng, Mubashara Akhtar, Rami Aly, Zhijiang Guo, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal, James Thorne, Andreas Vlachos
Venue:
FEVER
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
297–306
Language:
URL:
https://aclanthology.org/2024.fever-1.30
DOI:
Bibkey:
Cite (ACL):
Sushant Gautam and Roxana Pop. 2024. FactGenius: Combining Zero-Shot Prompting and Fuzzy Relation Mining to Improve Fact Verification with Knowledge Graphs. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pages 297–306, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
FactGenius: Combining Zero-Shot Prompting and Fuzzy Relation Mining to Improve Fact Verification with Knowledge Graphs (Gautam & Pop, FEVER 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.fever-1.30.pdf