“Is Hate Lost in Translation?”: Evaluation of Multilingual LGBTQIA+ Hate Speech Detection

Fai Leui Chan, Duke Nguyen, Aditya Joshi


Abstract
This paper explores the challenges of detecting LGBTQIA+ hate speech of large language models across multiple languages, including English, Italian, Chinese and (code-mixed) English-Tamil, examining the impact of machine translation and whether the nuances of hate speech are preserved across translation. We examine the hate speech detection ability of zero-shot and fine-tuned GPT. Our findings indicate that: (1) English has the highest performance and the code-mixing scenario of English-Tamil being the lowest, (2) fine-tuning improves performance consistently across languages whilst translation yields mixed results. Through simple experimentation with original text and machine-translated text for hate speech detection along with a qualitative error analysis, this paper sheds light on the socio-cultural nuances and complexities of languages that may not be captured by automatic translation.
Anthology ID:
2024.alta-1.11
Volume:
Proceedings of the 22nd Annual Workshop of the Australasian Language Technology Association
Month:
December
Year:
2024
Address:
Canberra, Australia
Editors:
Tim Baldwin, Sergio José Rodríguez Méndez, Nicholas Kuo
Venue:
ALTA
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
146–152
Language:
URL:
https://aclanthology.org/2024.alta-1.11/
DOI:
Bibkey:
Cite (ACL):
Fai Leui Chan, Duke Nguyen, and Aditya Joshi. 2024. “Is Hate Lost in Translation?”: Evaluation of Multilingual LGBTQIA+ Hate Speech Detection. In Proceedings of the 22nd Annual Workshop of the Australasian Language Technology Association, pages 146–152, Canberra, Australia. Association for Computational Linguistics.
Cite (Informal):
“Is Hate Lost in Translation?”: Evaluation of Multilingual LGBTQIA+ Hate Speech Detection (Chan et al., ALTA 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.alta-1.11.pdf