Is there Gender Bias in Dependency Parsing? Revisiting “Women’s Syntactic Resilience”

Paul Go, Agnieszka Falenska


Abstract
In this paper, we revisit the seminal work of Garimella et al. 2019, who reported that dependency parsers learn demographically-related signals from their training data and perform differently on sentences authored by people of different genders. We re-run all the parsing experiments from Garimella et al. 2019 and find that their results are not reproducible. Additionally, the original patterns suggesting the presence of gender biases fail to generalize to other treebank and parsing architecture. Instead, our data analysis uncovers methodological shortcomings in the initial study that artificially introduced differences into female and male datasets during preprocessing. These disparities potentially compromised the validity of the original conclusions.
Anthology ID:
2024.gebnlp-1.17
Volume:
Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Agnieszka Faleńska, Christine Basta, Marta Costa-jussà, Seraphina Goldfarb-Tarrant, Debora Nozza
Venues:
GeBNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
269–279
Language:
URL:
https://aclanthology.org/2024.gebnlp-1.17
DOI:
10.18653/v1/2024.gebnlp-1.17
Bibkey:
Cite (ACL):
Paul Go and Agnieszka Falenska. 2024. Is there Gender Bias in Dependency Parsing? Revisiting “Women’s Syntactic Resilience”. In Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 269–279, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Is there Gender Bias in Dependency Parsing? Revisiting “Women’s Syntactic Resilience” (Go & Falenska, GeBNLP-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.gebnlp-1.17.pdf