Fine-Grained Fairness Analysis of Abusive Language Detection Systems with CheckList

Marta Marchiori Manerba, Sara Tonelli


Abstract
Current abusive language detection systems have demonstrated unintended bias towards sensitive features such as nationality or gender. This is a crucial issue, which may harm minorities and underrepresented groups if such systems were integrated in real-world applications. In this paper, we create ad hoc tests through the CheckList tool (Ribeiro et al., 2020) to detect biases within abusive language classifiers for English. We compare the behaviour of two BERT-based models, one trained on a generic hate speech dataset and the other on a dataset for misogyny detection. Our evaluation shows that, although BERT-based classifiers achieve high accuracy levels on a variety of natural language processing tasks, they perform very poorly as regards fairness and bias, in particular on samples involving implicit stereotypes, expressions of hate towards minorities and protected attributes such as race or sexual orientation. We release both the notebooks implemented to extend the Fairness tests and the synthetic datasets usable to evaluate systems bias independently of CheckList.
Anthology ID:
2021.woah-1.9
Volume:
Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021)
Month:
August
Year:
2021
Address:
Online
Editors:
Aida Mostafazadeh Davani, Douwe Kiela, Mathias Lambert, Bertie Vidgen, Vinodkumar Prabhakaran, Zeerak Waseem
Venue:
WOAH
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
81–91
Language:
URL:
https://aclanthology.org/2021.woah-1.9
DOI:
10.18653/v1/2021.woah-1.9
Bibkey:
Cite (ACL):
Marta Marchiori Manerba and Sara Tonelli. 2021. Fine-Grained Fairness Analysis of Abusive Language Detection Systems with CheckList. In Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021), pages 81–91, Online. Association for Computational Linguistics.
Cite (Informal):
Fine-Grained Fairness Analysis of Abusive Language Detection Systems with CheckList (Manerba & Tonelli, WOAH 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.woah-1.9.pdf
Video:
 https://aclanthology.org/2021.woah-1.9.mp4
Code
 martamarchiori/fairness-analysis-with-checklist