CICL_DMS at SemEval-2023 Task 11: Learning With Disagreements (Le-Wi-Di)

Dennis Grötzinger, Simon Heuschkel, Matthias Drews


Abstract
In this system paper, we describe our submission for the 11th task of SemEval2023: Learning with Disagreements, or Le-Wi-Di for short. In the task, the assumption that there is a single gold label in NLP tasks such as hate speech or misogyny detection is challenged, and instead the opinions of multiple annotators are considered. The goal is instead to capture the agreements/disagreements of the annotators. For our system, we utilize the capabilities of modern large-language models as our backbone and investigate various techniques built on top, such as ensemble learning, multi-task learning, or Gaussian processes. Our final submission shows promising results and we achieve an upper-half finish.
Anthology ID:
2023.semeval-1.141
Volume:
Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Giovanni Da San Martino, Harish Tayyar Madabushi, Ritesh Kumar, Elisa Sartori
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
1030–1036
Language:
URL:
https://aclanthology.org/2023.semeval-1.141
DOI:
10.18653/v1/2023.semeval-1.141
Bibkey:
Cite (ACL):
Dennis Grötzinger, Simon Heuschkel, and Matthias Drews. 2023. CICL_DMS at SemEval-2023 Task 11: Learning With Disagreements (Le-Wi-Di). In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1030–1036, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
CICL_DMS at SemEval-2023 Task 11: Learning With Disagreements (Le-Wi-Di) (Grötzinger et al., SemEval 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.semeval-1.141.pdf