Mitigation of Diachronic Bias in Fake News Detection Dataset

Taichi Murayama, Shoko Wakamiya, Eiji Aramaki


Abstract
Fake news causes significant damage to society. To deal with these fake news, several studies on building detection models and arranging datasets have been conducted. Most of the fake news datasets depend on a specific time period. Consequently, the detection models trained on such a dataset have difficulty detecting novel fake news generated by political changes and social changes; they may possibly result in biased output from the input, including specific person names and organizational names. We refer to this problem as Diachronic Bias because it is caused by the creation date of news in each dataset. In this study, we confirm the bias, especially proper nouns including person names, from the deviation of phrase appearances in each dataset. Based on these findings, we propose masking methods using Wikidata to mitigate the influence of person names and validate whether they make fake news detection models robust through experiments with in-domain and out-of-domain data.
Anthology ID:
2021.wnut-1.21
Volume:
Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021)
Month:
November
Year:
2021
Address:
Online
Editors:
Wei Xu, Alan Ritter, Tim Baldwin, Afshin Rahimi
Venue:
WNUT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
182–188
Language:
URL:
https://aclanthology.org/2021.wnut-1.21
DOI:
10.18653/v1/2021.wnut-1.21
Bibkey:
Cite (ACL):
Taichi Murayama, Shoko Wakamiya, and Eiji Aramaki. 2021. Mitigation of Diachronic Bias in Fake News Detection Dataset. In Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021), pages 182–188, Online. Association for Computational Linguistics.
Cite (Informal):
Mitigation of Diachronic Bias in Fake News Detection Dataset (Murayama et al., WNUT 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.wnut-1.21.pdf