Demoting Racial Bias in Hate Speech Detection

Mengzhou Xia, Anjalie Field, Yulia Tsvetkov


Abstract
In the task of hate speech detection, there exists a high correlation between African American English (AAE) and annotators’ perceptions of toxicity in current datasets. This bias in annotated training data and the tendency of machine learning models to amplify it cause AAE text to often be mislabeled as abusive/offensive/hate speech (high false positive rate) by current hate speech classifiers. Here, we use adversarial training to mitigate this bias. Experimental results on one hate speech dataset and one AAE dataset suggest that our method is able to reduce the false positive rate for AAE text with only a minimal compromise on the performance of hate speech classification.
Anthology ID:
2020.socialnlp-1.2
Volume:
Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media
Month:
July
Year:
2020
Address:
Online
Venues:
ACL | SocialNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7–14
Language:
URL:
https://aclanthology.org/2020.socialnlp-1.2
DOI:
10.18653/v1/2020.socialnlp-1.2
Bibkey:
Cite (ACL):
Mengzhou Xia, Anjalie Field, and Yulia Tsvetkov. 2020. Demoting Racial Bias in Hate Speech Detection. In Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media, pages 7–14, Online. Association for Computational Linguistics.
Cite (Informal):
Demoting Racial Bias in Hate Speech Detection (Xia et al., SocialNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.socialnlp-1.2.pdf
Video:
 http://slideslive.com/38929902