FANG-COVID: A New Large-Scale Benchmark Dataset for Fake News Detection in German

Justus Mattern, Yu Qiao, Elma Kerz, Daniel Wiechmann, Markus Strohmaier


Abstract
As the world continues to fight the COVID-19 pandemic, it is simultaneously fighting an ‘infodemic’ – a flood of disinformation and spread of conspiracy theories leading to health threats and the division of society. To combat this infodemic, there is an urgent need for benchmark datasets that can help researchers develop and evaluate models geared towards automatic detection of disinformation. While there are increasing efforts to create adequate, open-source benchmark datasets for English, comparable resources are virtually unavailable for German, leaving research for the German language lagging significantly behind. In this paper, we introduce the new benchmark dataset FANG-COVID consisting of 28,056 real and 13,186 fake German news articles related to the COVID-19 pandemic as well as data on their propagation on Twitter. Furthermore, we propose an explainable textual- and social context-based model for fake news detection, compare its performance to “black-box” models and perform feature ablation to assess the relative importance of human-interpretable features in distinguishing fake news from authentic news.
Anthology ID:
2021.fever-1.9
Volume:
Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)
Month:
November
Year:
2021
Address:
Dominican Republic
Editors:
Rami Aly, Christos Christodoulopoulos, Oana Cocarascu, Zhijiang Guo, Arpit Mittal, Michael Schlichtkrull, James Thorne, Andreas Vlachos
Venue:
FEVER
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
78–91
Language:
URL:
https://aclanthology.org/2021.fever-1.9
DOI:
10.18653/v1/2021.fever-1.9
Bibkey:
Cite (ACL):
Justus Mattern, Yu Qiao, Elma Kerz, Daniel Wiechmann, and Markus Strohmaier. 2021. FANG-COVID: A New Large-Scale Benchmark Dataset for Fake News Detection in German. In Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER), pages 78–91, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
FANG-COVID: A New Large-Scale Benchmark Dataset for Fake News Detection in German (Mattern et al., FEVER 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.fever-1.9.pdf
Code
 justusmattern/fang-covid
Data
CoAID