Adam-Smith at SemEval-2023 Task 4: Discovering Human Values in Arguments with Ensembles of Transformer-based Models

Daniel Schroter, Daryna Dementieva, Georg Groh


Abstract
This paper presents the best-performing approach alias “Adam Smith” for the SemEval-2023 Task 4: “Identification of Human Values behind Arguments”. The goal of the task was to create systems that automatically identify the values within textual arguments. We train transformer-based models until they reach their loss minimum or f1-score maximum. Ensembling the models by selecting one global decision threshold that maximizes the f1-score leads to the best-performing system in the competition. Ensembling based on stacking with logistic regressions shows the best performance on an additional dataset provided to evaluate the robustness (“Nahj al-Balagha”). Apart from outlining the submitted system, we demonstrate that the use of the large ensemble model is not necessary and that the system size can be significantly reduced.
Anthology ID:
2023.semeval-1.74
Volume:
Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Giovanni Da San Martino, Harish Tayyar Madabushi, Ritesh Kumar, Elisa Sartori
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
532–541
Language:
URL:
https://aclanthology.org/2023.semeval-1.74
DOI:
10.18653/v1/2023.semeval-1.74
Bibkey:
Cite (ACL):
Daniel Schroter, Daryna Dementieva, and Georg Groh. 2023. Adam-Smith at SemEval-2023 Task 4: Discovering Human Values in Arguments with Ensembles of Transformer-based Models. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 532–541, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Adam-Smith at SemEval-2023 Task 4: Discovering Human Values in Arguments with Ensembles of Transformer-based Models (Schroter et al., SemEval 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.semeval-1.74.pdf
Video:
 https://aclanthology.org/2023.semeval-1.74.mp4