What social attitudes about gender does BERT encode? Leveraging insights from psycholinguistics

Julia Watson, Barend Beekhuizen, Suzanne Stevenson


Abstract
Much research has sought to evaluate the degree to which large language models reflect social biases. We complement such work with an approach to elucidating the connections between language model predictions and people’s social attitudes. We show how word preferences in a large language model reflect social attitudes about gender, using two datasets from human experiments that found differences in gendered or gender neutral word choices by participants with differing views on gender (progressive, moderate, or conservative). We find that the language model BERT takes into account factors that shape human lexical choice of such language, but may not weigh those factors in the same way people do. Moreover, we show that BERT’s predictions most resemble responses from participants with moderate to conservative views on gender. Such findings illuminate how a language model: (1) may differ from people in how it deploys words that signal gender, and (2) may prioritize some social attitudes over others.
Anthology ID:
2023.acl-long.375
Volume:
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6790–6809
Language:
URL:
https://aclanthology.org/2023.acl-long.375
DOI:
10.18653/v1/2023.acl-long.375
Bibkey:
Cite (ACL):
Julia Watson, Barend Beekhuizen, and Suzanne Stevenson. 2023. What social attitudes about gender does BERT encode? Leveraging insights from psycholinguistics. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6790–6809, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
What social attitudes about gender does BERT encode? Leveraging insights from psycholinguistics (Watson et al., ACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.acl-long.375.pdf
Video:
 https://aclanthology.org/2023.acl-long.375.mp4