Hell Hath No Fury? Correcting Bias in the NRC Emotion Lexicon

Samira Zad, Joshuan Jimenez, Mark Finlayson


Abstract
There have been several attempts to create an accurate and thorough emotion lexicon in English, which identifies the emotional content of words. Of the several commonly used resources, the NRC emotion lexicon (Mohammad and Turney, 2013b) has received the most attention due to its availability, size, and its choice of Plutchik’s expressive 8-class emotion model. In this paper we identify a large number of troubling entries in the NRC lexicon, where words that should in most contexts be emotionally neutral, with no affect (e.g., ‘lesbian’, ‘stone’, ‘mountain’), are associated with emotional labels that are inaccurate, nonsensical, pejorative, or, at best, highly contingent and context-dependent (e.g., ‘lesbian’ labeled as Disgust and Sadness, ‘stone’ as Anger, or ‘mountain’ as Anticipation). We describe a procedure for semi-automatically correcting these problems in the NRC, which includes disambiguating POS categories and aligning NRC entries with other emotion lexicons to infer the accuracy of labels. We demonstrate via an experimental benchmark that the quality of the resources is thus improved. We release the revised resource and our code to enable other researchers to reproduce and build upon results.
Anthology ID:
2021.woah-1.11
Volume:
Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021)
Month:
August
Year:
2021
Address:
Online
Editors:
Aida Mostafazadeh Davani, Douwe Kiela, Mathias Lambert, Bertie Vidgen, Vinodkumar Prabhakaran, Zeerak Waseem
Venue:
WOAH
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
102–113
Language:
URL:
https://aclanthology.org/2021.woah-1.11
DOI:
10.18653/v1/2021.woah-1.11
Bibkey:
Cite (ACL):
Samira Zad, Joshuan Jimenez, and Mark Finlayson. 2021. Hell Hath No Fury? Correcting Bias in the NRC Emotion Lexicon. In Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021), pages 102–113, Online. Association for Computational Linguistics.
Cite (Informal):
Hell Hath No Fury? Correcting Bias in the NRC Emotion Lexicon (Zad et al., WOAH 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.woah-1.11.pdf
Video:
 https://aclanthology.org/2021.woah-1.11.mp4
Data
ISEAR