Characterizing English Variation across Social Media Communities with BERT

Li Lucy, David Bamman


Abstract
Much previous work characterizing language variation across Internet social groups has focused on the types of words used by these groups. We extend this type of study by employing BERT to characterize variation in the senses of words as well, analyzing two months of English comments in 474 Reddit communities. The specificity of different sense clusters to a community, combined with the specificity of a community’s unique word types, is used to identify cases where a social group’s language deviates from the norm. We validate our metrics using user-created glossaries and draw on sociolinguistic theories to connect language variation with trends in community behavior. We find that communities with highly distinctive language are medium-sized, and their loyal and highly engaged users interact in dense networks.
Anthology ID:
2021.tacl-1.33
Volume:
Transactions of the Association for Computational Linguistics, Volume 9
Month:
Year:
2021
Address:
Cambridge, MA
Editors:
Brian Roark, Ani Nenkova
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
538–556
Language:
URL:
https://aclanthology.org/2021.tacl-1.33
DOI:
10.1162/tacl_a_00383
Bibkey:
Cite (ACL):
Li Lucy and David Bamman. 2021. Characterizing English Variation across Social Media Communities with BERT. Transactions of the Association for Computational Linguistics, 9:538–556.
Cite (Informal):
Characterizing English Variation across Social Media Communities with BERT (Lucy & Bamman, TACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.tacl-1.33.pdf
Video:
 https://aclanthology.org/2021.tacl-1.33.mp4