What Did You Learn To Hate? A Topic-Oriented Analysis of Generalization in Hate Speech Detection

Tom Bourgeade, Patricia Chiril, Farah Benamara, Véronique Moriceau


Abstract
Hate speech has unfortunately become a significant phenomenon on social media platforms, and it can cover various topics (misogyny, sexism, racism, xenophobia, etc.) and targets (e.g., black people, women). Various hate speech detection datasets have been proposed, some annotated for specific topics, and others for hateful speech in general. In either case, they often employ different annotation guidelines, which can lead to inconsistencies, even in datasets focusing on the same topics. This can cause issues in models trying to generalize across more data and more topics in order to improve detection accuracy. In this paper, we propose, for the first time, a topic-oriented approach to study generalization across popular hate speech datasets. We first perform a comparative analysis of the performances of Transformer-based models in capturing topic-generic and topic-specific knowledge when trained on different datasets. We then propose a novel, simple yet effective approach to study more precisely which topics are best captured in implicit manifestations of hate, showing that selecting combinations of datasets with better out-of-domain topical coverage improves the reliability of automatic hate speech detection.
Anthology ID:
2023.eacl-main.254
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3495–3508
Language:
URL:
https://aclanthology.org/2023.eacl-main.254
DOI:
10.18653/v1/2023.eacl-main.254
Bibkey:
Cite (ACL):
Tom Bourgeade, Patricia Chiril, Farah Benamara, and Véronique Moriceau. 2023. What Did You Learn To Hate? A Topic-Oriented Analysis of Generalization in Hate Speech Detection. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3495–3508, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
What Did You Learn To Hate? A Topic-Oriented Analysis of Generalization in Hate Speech Detection (Bourgeade et al., EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-main.254.pdf
Video:
 https://aclanthology.org/2023.eacl-main.254.mp4