Features or Spurious Artifacts? Data-centric Baselines for Fair and Robust Hate Speech Detection

Alan Ramponi, Sara Tonelli


Abstract
Avoiding to rely on dataset artifacts to predict hate speech is at the cornerstone of robust and fair hate speech detection. In this paper we critically analyze lexical biases in hate speech detection via a cross-platform study, disentangling various types of spurious and authentic artifacts and analyzing their impact on out-of-distribution fairness and robustness. We experiment with existing approaches and propose simple yet surprisingly effective data-centric baselines. Our results on English data across four platforms show that distinct spurious artifacts require different treatments to ultimately attain both robustness and fairness in hate speech detection. To encourage research in this direction, we release all baseline models and the code to compute artifacts, pointing it out as a complementary and necessary addition to the data statements practice.
Anthology ID:
2022.naacl-main.221
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3027–3040
Language:
URL:
https://aclanthology.org/2022.naacl-main.221
DOI:
10.18653/v1/2022.naacl-main.221
Bibkey:
Cite (ACL):
Alan Ramponi and Sara Tonelli. 2022. Features or Spurious Artifacts? Data-centric Baselines for Fair and Robust Hate Speech Detection. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3027–3040, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
Features or Spurious Artifacts? Data-centric Baselines for Fair and Robust Hate Speech Detection (Ramponi & Tonelli, NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-main.221.pdf
Video:
 https://aclanthology.org/2022.naacl-main.221.mp4
Code
 dhfbk/hate-speech-artifacts
Data
Hate Speech