Span Identification of Epistemic Stance-Taking in Academic Written English

Masaki Eguchi, Kristopher Kyle


Abstract
Responding to the increasing need for automated writing evaluation (AWE) systems to assess language use beyond lexis and grammar (Burstein et al., 2016), we introduce a new approach to identify rhetorical features of stance in academic English writing. Drawing on the discourse-analytic framework of engagement in the Appraisal analysis (Martin & White, 2005), we manually annotated 4,688 sentences (126,411 tokens) for eight rhetorical stance categories (e.g., PROCLAIM, ATTRIBUTION) and additional discourse elements. We then report an experiment to train machine learning models to identify and categorize the spans of these stance expressions. The best-performing model (RoBERTa + LSTM) achieved macro-averaged F1 of .7208 in the span identification of stance-taking expressions, slightly outperforming the intercoder reliability estimates before adjudication (F1 = .6629).
Anthology ID:
2023.bea-1.35
Volume:
Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Ekaterina Kochmar, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Nitin Madnani, Anaïs Tack, Victoria Yaneva, Zheng Yuan, Torsten Zesch
Venue:
BEA
SIG:
SIGEDU
Publisher:
Association for Computational Linguistics
Note:
Pages:
429–442
Language:
URL:
https://aclanthology.org/2023.bea-1.35
DOI:
10.18653/v1/2023.bea-1.35
Bibkey:
Cite (ACL):
Masaki Eguchi and Kristopher Kyle. 2023. Span Identification of Epistemic Stance-Taking in Academic Written English. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 429–442, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Span Identification of Epistemic Stance-Taking in Academic Written English (Eguchi & Kyle, BEA 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.bea-1.35.pdf