Using Analytic Scoring Rubrics in the Automatic Assessment of College-Level Summary Writing Tasks in L2

Tamara Sladoljev-Agejev, Jan Šnajder


Abstract
Assessing summaries is a demanding, yet useful task which provides valuable information on language competence, especially for second language learners. We consider automated scoring of college-level summary writing task in English as a second language (EL2). We adopt the Reading-for-Understanding (RU) cognitive framework, extended with the Reading-to-Write (RW) element, and use analytic scoring with six rubrics covering content and writing quality. We show that regression models with reference-based and linguistic features considerably outperform the baselines across all the rubrics. Moreover, we find interesting correlations between summary features and analytic rubrics, revealing the links between the RU and RW constructs.
Anthology ID:
I17-2031
Volume:
Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Month:
November
Year:
2017
Address:
Taipei, Taiwan
Venue:
IJCNLP
SIG:
Publisher:
Asian Federation of Natural Language Processing
Note:
Pages:
181–186
Language:
URL:
https://aclanthology.org/I17-2031
DOI:
Bibkey:
Cite (ACL):
Tamara Sladoljev-Agejev and Jan Šnajder. 2017. Using Analytic Scoring Rubrics in the Automatic Assessment of College-Level Summary Writing Tasks in L2. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 181–186, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Cite (Informal):
Using Analytic Scoring Rubrics in the Automatic Assessment of College-Level Summary Writing Tasks in L2 (Sladoljev-Agejev & Šnajder, IJCNLP 2017)
Copy Citation:
PDF:
https://aclanthology.org/I17-2031.pdf