An Expert Annotated Dataset for the Detection of Online Misogyny
Ella Guest | Bertie Vidgen | Alexandros Mittos | Nishanth Sastry | Gareth Tyson | Helen Margetts
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
Online misogyny is a pernicious social problem that risks making online platforms toxic and unwelcoming to women. We present a new hierarchical taxonomy for online misogyny, as well as an expert labelled dataset to enable automatic classification of misogynistic content. The dataset consists of 6567 labels for Reddit posts and comments. As previous research has found untrained crowdsourced annotators struggle with identifying misogyny, we hired and trained annotators and provided them with robust annotation guidelines. We report baseline classification performance on the binary classification task, achieving accuracy of 0.93 and F1 of 0.43. The codebook and datasets are made freely available for future researchers.