Overview of the 2022 Validity and Novelty Prediction Shared Task

Philipp Heinisch, Anette Frank, Juri Opitz, Moritz Plenz, Philipp Cimiano


Abstract
This paper provides an overview of the Argument Validity and Novelty Prediction Shared Task that was organized as part of the 9th Workshop on Argument Mining (ArgMining 2022). The task focused on the prediction of the validity and novelty of a conclusion given a textual premise. Validity is defined as the degree to which the conclusion is justified with respect to the given premise. Novelty defines the degree to which the conclusion contains content that is new in relation to the premise. Six groups participated in the task, submitting overall 13 system runs for the subtask of binary classification and 2 system runs for the subtask of relative classification. The results reveal that the task is challenging, with best results obtained for Validity prediction in the range of 75% F1 score, for Novelty prediction of 70% F1 score and for correctly predicting both Validity and Novelty of 45% F1 score. In this paper we summarize the task definition and dataset. We give an overview of the results obtained by the participating systems, as well as insights to be gained from the diverse contributions.
Anthology ID:
2022.argmining-1.7
Volume:
Proceedings of the 9th Workshop on Argument Mining
Month:
October
Year:
2022
Address:
Online and in Gyeongju, Republic of Korea
Venue:
ArgMining
SIG:
Publisher:
International Conference on Computational Linguistics
Note:
Pages:
84–94
Language:
URL:
https://aclanthology.org/2022.argmining-1.7
DOI:
Bibkey:
Cite (ACL):
Philipp Heinisch, Anette Frank, Juri Opitz, Moritz Plenz, and Philipp Cimiano. 2022. Overview of the 2022 Validity and Novelty Prediction Shared Task. In Proceedings of the 9th Workshop on Argument Mining, pages 84–94, Online and in Gyeongju, Republic of Korea. International Conference on Computational Linguistics.
Cite (Informal):
Overview of the 2022 Validity and Novelty Prediction Shared Task (Heinisch et al., ArgMining 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.argmining-1.7.pdf
Data
ConceptNet