AMREx: AMR for Explainable Fact Verification

Chathuri Jayaweera, Sangpil Youm, Bonnie J Dorr


Abstract
With the advent of social media networks and the vast amount of information circulating through them, automatic fact verification is an essential component to prevent the spread of misinformation. It is even more useful to have fact verification systems that provide explanations along with their classifications to ensure accurate predictions. To address both of these requirements, we implement AMREx, an Abstract Meaning Representation (AMR)-based veracity prediction and explanation system for fact verification using a combination of Smatch, an AMR evaluation metric to measure meaning containment and textual similarity, and demonstrate its effectiveness in producing partially explainable justifications using two community standard fact verification datasets, FEVER and AVeriTeC. AMREx surpasses the AVeriTec baseline accuracy showing the effectiveness of our approach for real-world claim verification. It follows an interpretable pipeline and returns an explainable AMR node mapping to clarify the system’s veracity predictions when applicable. We further demonstrate that AMREx output can be used to prompt LLMs to generate natural-language explanations using the AMR mappings as a guide to lessen the probability of hallucinations.
Anthology ID:
2024.fever-1.26
Volume:
Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER)
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse, Zhenyun Deng, Mubashara Akhtar, Rami Aly, Zhijiang Guo, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal, James Thorne, Andreas Vlachos
Venue:
FEVER
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
234–244
Language:
URL:
https://aclanthology.org/2024.fever-1.26
DOI:
10.18653/v1/2024.fever-1.26
Bibkey:
Cite (ACL):
Chathuri Jayaweera, Sangpil Youm, and Bonnie J Dorr. 2024. AMREx: AMR for Explainable Fact Verification. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pages 234–244, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
AMREx: AMR for Explainable Fact Verification (Jayaweera et al., FEVER 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.fever-1.26.pdf