Analysis of Annotator Demographics in Sexism Detection

Narjes Tahaei, Sabine Bergler


Abstract
This study explores the effect of annotators’ demographic features on labeling sexist content in social media datasets, specifically focusing on the EXIST dataset, which includes direct sexist messages, reports and descriptions of sexist experiences and stereotypes. We investigate how various demographic backgrounds influence annotation outcomes and examine methods to incorporate these features into BERT-based model training. Our experiments demonstrate that adding demographic information improves performance in detecting sexism and assessing intention of the author.
Anthology ID:
2024.gebnlp-1.24
Volume:
Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Agnieszka Faleńska, Christine Basta, Marta Costa-jussà, Seraphina Goldfarb-Tarrant, Debora Nozza
Venues:
GeBNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
376–383
Language:
URL:
https://aclanthology.org/2024.gebnlp-1.24
DOI:
Bibkey:
Cite (ACL):
Narjes Tahaei and Sabine Bergler. 2024. Analysis of Annotator Demographics in Sexism Detection. In Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 376–383, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Analysis of Annotator Demographics in Sexism Detection (Tahaei & Bergler, GeBNLP-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.gebnlp-1.24.pdf