Implicit Phenomena in Short-answer Scoring Data

Marie Bexte, Andrea Horbach, Torsten Zesch


Abstract
Short-answer scoring is the task of assessing the correctness of a short text given as response to a question that can come from a variety of educational scenarios. As only content, not form, is important, the exact wording including the explicitness of an answer should not matter. However, many state-of-the-art scoring models heavily rely on lexical information, be it word embeddings in a neural network or n-grams in an SVM. Thus, the exact wording of an answer might very well make a difference. We therefore quantify to what extent implicit language phenomena occur in short answer datasets and examine the influence they have on automatic scoring performance. We find that the level of implicitness depends on the individual question, and that some phenomena are very frequent. Resolving implicit wording to explicit formulations indeed tends to improve automatic scoring performance.
Anthology ID:
2021.unimplicit-1.2
Volume:
Proceedings of the 1st Workshop on Understanding Implicit and Underspecified Language
Month:
August
Year:
2021
Address:
Online
Editors:
Michael Roth, Reut Tsarfaty, Yoav Goldberg
Venue:
unimplicit
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11–19
Language:
URL:
https://aclanthology.org/2021.unimplicit-1.2
DOI:
10.18653/v1/2021.unimplicit-1.2
Bibkey:
Cite (ACL):
Marie Bexte, Andrea Horbach, and Torsten Zesch. 2021. Implicit Phenomena in Short-answer Scoring Data. In Proceedings of the 1st Workshop on Understanding Implicit and Underspecified Language, pages 11–19, Online. Association for Computational Linguistics.
Cite (Informal):
Implicit Phenomena in Short-answer Scoring Data (Bexte et al., unimplicit 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.unimplicit-1.2.pdf