Decoding Ableism in Large Language Models: An Intersectional Approach

Rong Li, Ashwini Kamaraj, Jing Ma, Sarah Ebling


Abstract
With the pervasive use of large language models (LLMs) across various domains, addressing the inherent ableist biases within these models requires more attention and resolution. This paper examines ableism in three LLMs (GPT-3.5, GPT-4, and Llama 3) by analyzing the intersection of disability with two additional social categories: gender and social class. Utilizing two task-specific prompts, we generated and analyzed text outputs with two metrics, VADER and regard, to evaluate sentiment and social perception biases within the responses. Our results indicate a marked improvement in bias mitigation from GPT-3.5 to GPT-4, with the latter demonstrating more positive sentiments overall, while Llama 3 showed comparatively weaker performance. Additionally, our findings underscore the complexity of intersectional biases: These biases are shaped by the combined effects of disability, gender, and class, which alter the expression and perception of ableism in LLM outputs. This research highlights the necessity for more nuanced and inclusive bias mitigation strategies in AI development, contributing to the ongoing dialogue on ethical AI practices.
Anthology ID:
2024.nlp4pi-1.22
Volume:
Proceedings of the Third Workshop on NLP for Positive Impact
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Daryna Dementieva, Oana Ignat, Zhijing Jin, Rada Mihalcea, Giorgio Piatti, Joel Tetreault, Steven Wilson, Jieyu Zhao
Venue:
NLP4PI
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
232–249
Language:
URL:
https://aclanthology.org/2024.nlp4pi-1.22
DOI:
Bibkey:
Cite (ACL):
Rong Li, Ashwini Kamaraj, Jing Ma, and Sarah Ebling. 2024. Decoding Ableism in Large Language Models: An Intersectional Approach. In Proceedings of the Third Workshop on NLP for Positive Impact, pages 232–249, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Decoding Ableism in Large Language Models: An Intersectional Approach (Li et al., NLP4PI 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.nlp4pi-1.22.pdf