Eyes Don’t Lie: Subjective Hate Annotation and Detection with Gaze

Özge Alacam, Sanne Hoeken, Sina Zarrieß


Abstract
Hate speech is a complex and subjective phenomenon. In this paper, we present a dataset (GAZE4HATE) that provides gaze data collected in a hate speech annotation experiment. We study whether the gaze of an annotator provides predictors of their subjective hatefulness rating, and how gaze features can improve Hate Speech Detection (HSD). We conduct experiments on statistical modeling of subjective hate ratings and gaze and analyze to what extent rationales derived from hate speech models correspond to human gaze and explanations in our data. Finally, we introduce MEANION, a first gaze-integrated HSD model. Our experiments show that particular gaze features like dwell time or fixation counts systematically correlate with annotators’ subjective hate ratings and improve predictions of text-only hate speech models.
Anthology ID:
2024.emnlp-main.11
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
187–205
Language:
URL:
https://aclanthology.org/2024.emnlp-main.11
DOI:
Bibkey:
Cite (ACL):
Özge Alacam, Sanne Hoeken, and Sina Zarrieß. 2024. Eyes Don’t Lie: Subjective Hate Annotation and Detection with Gaze. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 187–205, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Eyes Don’t Lie: Subjective Hate Annotation and Detection with Gaze (Alacam et al., EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-main.11.pdf