Machine-Aided Annotation for Fine-Grained Proposition Types in Argumentation

Yohan Jo, Elijah Mayfield, Chris Reed, Eduard Hovy


Abstract
We introduce a corpus of the 2016 U.S. presidential debates and commentary, containing 4,648 argumentative propositions annotated with fine-grained proposition types. Modern machine learning pipelines for analyzing argument have difficulty distinguishing between types of propositions based on their factuality, rhetorical positioning, and speaker commitment. Inability to properly account for these facets leaves such systems inaccurate in understanding of fine-grained proposition types. In this paper, we demonstrate an approach to annotating for four complex proposition types, namely normative claims, desires, future possibility, and reported speech. We develop a hybrid machine learning and human workflow for annotation that allows for efficient and reliable annotation of complex linguistic phenomena, and demonstrate with preliminary analysis of rhetorical strategies and structure in presidential debates. This new dataset and method can support technical researchers seeking more nuanced representations of argument, as well as argumentation theorists developing new quantitative analyses.
Anthology ID:
2020.lrec-1.127
Volume:
Proceedings of the 12th Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
1008–1018
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.127
DOI:
Bibkey:
Cite (ACL):
Yohan Jo, Elijah Mayfield, Chris Reed, and Eduard Hovy. 2020. Machine-Aided Annotation for Fine-Grained Proposition Types in Argumentation. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 1008–1018, Marseille, France. European Language Resources Association.
Cite (Informal):
Machine-Aided Annotation for Fine-Grained Proposition Types in Argumentation (Jo et al., LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.127.pdf