MIND at SemEval-2023 Task 11: From Uncertain Predictions to Subjective Disagreement

Giulia Rizzi, Alessandro Astorino, Daniel Scalena, Paolo Rosso, Elisabetta Fersini


Abstract
This paper describes the participation of the research laboratory MIND, at the University of Milano-Bicocca, in the SemEval 2023 task related to Learning With Disagreements (Le-Wi-Di). The main goal is to identify the level of agreement/disagreement from a collection of textual datasets with different characteristics in terms of style, language and task. The proposed approach is grounded on the hypothesis that the disagreement between annotators could be grasped by the uncertainty that a model, based on several linguistic characteristics, could have on the prediction of a given gold label.
Anthology ID:
2023.semeval-1.77
Volume:
Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Giovanni Da San Martino, Harish Tayyar Madabushi, Ritesh Kumar, Elisa Sartori
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
556–564
Language:
URL:
https://aclanthology.org/2023.semeval-1.77
DOI:
10.18653/v1/2023.semeval-1.77
Bibkey:
Cite (ACL):
Giulia Rizzi, Alessandro Astorino, Daniel Scalena, Paolo Rosso, and Elisabetta Fersini. 2023. MIND at SemEval-2023 Task 11: From Uncertain Predictions to Subjective Disagreement. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 556–564, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
MIND at SemEval-2023 Task 11: From Uncertain Predictions to Subjective Disagreement (Rizzi et al., SemEval 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.semeval-1.77.pdf
Video:
 https://aclanthology.org/2023.semeval-1.77.mp4