Emotional RobBERT and Insensitive BERTje: Combining Transformers and Affect Lexica for Dutch Emotion Detection

Luna De Bruyne, Orphee De Clercq, Veronique Hoste


Abstract
In a first step towards improving Dutch emotion detection, we try to combine the Dutch transformer models BERTje and RobBERT with lexicon-based methods. We propose two architectures: one in which lexicon information is directly injected into the transformer model and a meta-learning approach where predictions from transformers are combined with lexicon features. The models are tested on 1,000 Dutch tweets and 1,000 captions from TV-shows which have been manually annotated with emotion categories and dimensions. We find that RobBERT clearly outperforms BERTje, but that directly adding lexicon information to transformers does not improve performance. In the meta-learning approach, lexicon information does have a positive effect on BERTje, but not on RobBERT. This suggests that more emotional information is already contained within this latter language model.
Anthology ID:
2021.wassa-1.27
Volume:
Proceedings of the Eleventh Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis
Month:
April
Year:
2021
Address:
Online
Venues:
EACL | WASSA
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
257–263
Language:
URL:
https://aclanthology.org/2021.wassa-1.27
DOI:
Bibkey:
Cite (ACL):
Luna De Bruyne, Orphee De Clercq, and Veronique Hoste. 2021. Emotional RobBERT and Insensitive BERTje: Combining Transformers and Affect Lexica for Dutch Emotion Detection. In Proceedings of the Eleventh Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 257–263, Online. Association for Computational Linguistics.
Cite (Informal):
Emotional RobBERT and Insensitive BERTje: Combining Transformers and Affect Lexica for Dutch Emotion Detection (De Bruyne et al., WASSA 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.wassa-1.27.pdf