Bias at a Second Glance: A Deep Dive into Bias for German Educational Peer-Review Data Modeling

Thiemo Wambsganss, Vinitra Swamy, Roman Rietsche, Tanja Käser


Abstract
Natural Language Processing (NLP) has become increasingly utilized to provide adaptivity in educational applications. However, recent research has highlighted a variety of biases in pre-trained language models. While existing studies investigate bias in different domains, they are limited in addressing fine-grained analysis on educational corpora and text that is not English. In this work, we analyze bias across text and through multiple architectures on a corpus of 9,165 German peer-reviews collected from university students over five years. Notably, our corpus includes labels such as helpfulness, quality, and critical aspect ratings from the peer-review recipient as well as demographic attributes. We conduct a Word Embedding Association Test (WEAT) analysis on (1) our collected corpus in connection with the clustered labels, (2) the most common pre-trained German language models (T5, BERT, and GPT-2) and GloVe embeddings, and (3) the language models after fine-tuning on our collected data-set. In contrast to our initial expectations, we found that our collected corpus does not reveal many biases in the co-occurrence analysis or in the GloVe embeddings. However, the pre-trained German language models find substantial conceptual, racial, and gender bias and have significant changes in bias across conceptual and racial axes during fine-tuning on the peer-review data. With our research, we aim to contribute to the fourth UN sustainability goal (quality education) with a novel dataset, an understanding of biases in natural language education data, and the potential harms of not counteracting biases in language models for educational tasks.
Anthology ID:
2022.coling-1.115
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
1344–1356
Language:
URL:
https://aclanthology.org/2022.coling-1.115
DOI:
Bibkey:
Cite (ACL):
Thiemo Wambsganss, Vinitra Swamy, Roman Rietsche, and Tanja Käser. 2022. Bias at a Second Glance: A Deep Dive into Bias for German Educational Peer-Review Data Modeling. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1344–1356, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
Bias at a Second Glance: A Deep Dive into Bias for German Educational Peer-Review Data Modeling (Wambsganss et al., COLING 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.coling-1.115.pdf
Code
 epfl-ml4ed/bias-at-a-second-glance