Fine-grained essay scoring of a complex writing task for native speakers

Andrea Horbach, Dirk Scholten-Akoun, Yuning Ding, Torsten Zesch


Abstract
Automatic essay scoring is nowadays successfully used even in high-stakes tests, but this is mainly limited to holistic scoring of learner essays. We present a new dataset of essays written by highly proficient German native speakers that is scored using a fine-grained rubric with the goal to provide detailed feedback. Our experiments with two state-of-the-art scoring systems (a neural and a SVM-based one) show a large drop in performance compared to existing datasets. This demonstrates the need for such datasets that allow to guide research on more elaborate essay scoring methods.
Anthology ID:
W17-5040
Volume:
Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications
Month:
September
Year:
2017
Address:
Copenhagen, Denmark
Editors:
Joel Tetreault, Jill Burstein, Claudia Leacock, Helen Yannakoudakis
Venue:
BEA
SIG:
SIGEDU
Publisher:
Association for Computational Linguistics
Note:
Pages:
357–366
Language:
URL:
https://aclanthology.org/W17-5040
DOI:
10.18653/v1/W17-5040
Bibkey:
Cite (ACL):
Andrea Horbach, Dirk Scholten-Akoun, Yuning Ding, and Torsten Zesch. 2017. Fine-grained essay scoring of a complex writing task for native speakers. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, pages 357–366, Copenhagen, Denmark. Association for Computational Linguistics.
Cite (Informal):
Fine-grained essay scoring of a complex writing task for native speakers (Horbach et al., BEA 2017)
Copy Citation:
PDF:
https://aclanthology.org/W17-5040.pdf