Adaptive Axes: A Pipeline for In-domain Social Stereotype Analysis

Qingcheng Zeng, Mingyu Jin, Rob Voigt


Abstract
Prior work has explored the possibility of using the semantic information obtained from embedding representations to quantify social stereotypes, leveraging techniques such as word embeddings combined with a list of traits (Garg et al., 2018; Charlesworth et al., 2022) or semantic axes (An et al., 2018; Lucy et al., 2022). However, these approaches have struggled to fully capture the variability in stereotypes across different conceptual domains for the same social group (e.g., black in science, health, and art), in part because the identity of a word and the associations formed during pre-training can dominate its contextual representation (Field and Tsvetkov, 2019). This study explores the ability to recover stereotypes from the contexts surrounding targeted entities by utilizing state-of-the-art text embedding models and adaptive semantic axes enhanced by large language models (LLMs). Our results indicate that the proposed pipeline not only surpasses token-based methods in capturing in-domain framing but also effectively tracks stereotypes over time and along domain-specific semantic axes for in-domain texts. Our research highlights the potential of employing text embedding models to achieve a deeper understanding of nuanced social stereotypes.
Anthology ID:
2024.emnlp-main.872
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
15576–15593
Language:
URL:
https://aclanthology.org/2024.emnlp-main.872
DOI:
Bibkey:
Cite (ACL):
Qingcheng Zeng, Mingyu Jin, and Rob Voigt. 2024. Adaptive Axes: A Pipeline for In-domain Social Stereotype Analysis. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15576–15593, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Adaptive Axes: A Pipeline for In-domain Social Stereotype Analysis (Zeng et al., EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-main.872.pdf