Improving Explainable Fact-Checking via Sentence-Level Factual Reasoning

Francielle Vargas, Isadora Salles, Diego Alves, Ameeta Agrawal, Thiago A. S. Pardo, Fabrício Benevenuto


Abstract
Most existing fact-checking systems are unable to explain their decisions by providing relevant rationales (justifications) for their predictions. It highlights a lack of transparency that poses significant risks, such as the prevalence of unexpected biases, which may increase political polarization due to limitations in impartiality. To address this critical gap, we introduce SEntence-Level FActual Reasoning (SELFAR), aimed at improving explainable fact-checking. SELFAR relies on fact extraction and verification by predicting the news source reliability and factuality (veracity) of news articles or claims at the sentence level, generating post-hoc explanations using SHAP/LIME and zero-shot prompts. Our experiments show that unreliable news stories predominantly consist of subjective statements, in contrast to reliable ones. Consequently, predicting unreliable news articles at the sentence level by analyzing impartiality and subjectivity is a promising approach for fact extraction and improving explainable fact-checking. Furthermore, LIME outperforms SHAP in explaining predictions on reliability. Additionally, while zero-shot prompts provide highly readable explanations and achieve an accuracy of 0.71 in predicting factuality, their tendency to hallucinate remains a challenge. Lastly, this paper also presents the first study on explainable fact-checking in the Portuguese language.
Anthology ID:
2024.fever-1.23
Volume:
Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER)
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse, Zhenyun Deng, Mubashara Akhtar, Rami Aly, Zhijiang Guo, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal, James Thorne, Andreas Vlachos
Venue:
FEVER
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
192–204
Language:
URL:
https://aclanthology.org/2024.fever-1.23
DOI:
Bibkey:
Cite (ACL):
Francielle Vargas, Isadora Salles, Diego Alves, Ameeta Agrawal, Thiago A. S. Pardo, and Fabrício Benevenuto. 2024. Improving Explainable Fact-Checking via Sentence-Level Factual Reasoning. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pages 192–204, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Improving Explainable Fact-Checking via Sentence-Level Factual Reasoning (Vargas et al., FEVER 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.fever-1.23.pdf