Explainable Tsetlin Machine Framework for Fake News Detection with Credibility Score Assessment

Bimal Bhattarai, Ole-Christoffer Granmo, Lei Jiao


Abstract
The proliferation of fake news, i.e., news intentionally spread for misinformation, poses a threat to individuals and society. Despite various fact-checking websites such as PolitiFact, robust detection techniques are required to deal with the increase in fake news. Several deep learning models show promising results for fake news classification, however, their black-box nature makes it difficult to explain their classification decisions and quality-assure the models. We here address this problem by proposing a novel interpretable fake news detection framework based on the recently introduced Tsetlin Machine (TM). In brief, we utilize the conjunctive clauses of the TM to capture lexical and semantic properties of both true and fake news text. Further, we use clause ensembles to calculate the credibility of fake news. For evaluation, we conduct experiments on two publicly available datasets, PolitiFact and GossipCop, and demonstrate that the TM framework significantly outperforms previously published baselines by at least 5% in terms of accuracy, with the added benefit of an interpretable logic-based representation. In addition, our approach provides a higher F1-score than BERT and XLNet, however, we obtain slightly lower accuracy. We finally present a case study on our model’s explainability, demonstrating how it decomposes into meaningful words and their negations.
Anthology ID:
2022.lrec-1.523
Volume:
Proceedings of the Thirteenth Language Resources and Evaluation Conference
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
4894–4903
Language:
URL:
https://aclanthology.org/2022.lrec-1.523
DOI:
Bibkey:
Cite (ACL):
Bimal Bhattarai, Ole-Christoffer Granmo, and Lei Jiao. 2022. Explainable Tsetlin Machine Framework for Fake News Detection with Credibility Score Assessment. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4894–4903, Marseille, France. European Language Resources Association.
Cite (Informal):
Explainable Tsetlin Machine Framework for Fake News Detection with Credibility Score Assessment (Bhattarai et al., LREC 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.lrec-1.523.pdf
Code
 additional community code