Social Bias in Elicited Natural Language Inferences

Rachel Rudinger, Chandler May, Benjamin Van Durme


Abstract
We analyze the Stanford Natural Language Inference (SNLI) corpus in an investigation of bias and stereotyping in NLP data. The SNLI human-elicitation protocol makes it prone to amplifying bias and stereotypical associations, which we demonstrate statistically (using pointwise mutual information) and with qualitative examples.
Anthology ID:
W17-1609
Volume:
Proceedings of the First ACL Workshop on Ethics in Natural Language Processing
Month:
April
Year:
2017
Address:
Valencia, Spain
Editors:
Dirk Hovy, Shannon Spruit, Margaret Mitchell, Emily M. Bender, Michael Strube, Hanna Wallach
Venue:
EthNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
74–79
Language:
URL:
https://aclanthology.org/W17-1609
DOI:
10.18653/v1/W17-1609
Bibkey:
Cite (ACL):
Rachel Rudinger, Chandler May, and Benjamin Van Durme. 2017. Social Bias in Elicited Natural Language Inferences. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 74–79, Valencia, Spain. Association for Computational Linguistics.
Cite (Informal):
Social Bias in Elicited Natural Language Inferences (Rudinger et al., EthNLP 2017)
Copy Citation:
PDF:
https://aclanthology.org/W17-1609.pdf
Code
 cjmay/snli-ethics
Data
SNLI