When Shallow is Good Enough: Automatic Assessment of Conceptual Text Complexity using Shallow Semantic Features

Sanja Stajner, Ioana Hulpuș


Abstract
According to psycholinguistic studies, the complexity of concepts used in a text and the relations between mentioned concepts play the most important role in text understanding and maintaining reader’s interest. However, the classical approaches to automatic assessment of text complexity, and their commercial applications, take into consideration mainly syntactic and lexical complexity. Recently, we introduced the task of automatic assessment of conceptual text complexity, proposing a set of graph-based deep semantic features using DBpedia as a proxy to human knowledge. Given that such graphs can be noisy, incomplete, and computationally expensive to deal with, in this paper, we propose the use of textual features and shallow semantic features that only require entity linking. We compare the results obtained with new features with those of the state-of-the-art deep semantic features on two tasks: (1) pairwise comparison of two versions of the same text; and (2) five-level classification of texts. We find that the shallow features achieve state-of-the-art results on both tasks, significantly outperforming performances of the deep semantic features on the five-level classification task. Interestingly, the combination of the shallow and deep semantic features lead to a significant improvement of the performances on that task.
Anthology ID:
2020.lrec-1.177
Volume:
Proceedings of the 12th Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
1414–1422
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.177
DOI:
Bibkey:
Cite (ACL):
Sanja Stajner and Ioana Hulpuș. 2020. When Shallow is Good Enough: Automatic Assessment of Conceptual Text Complexity using Shallow Semantic Features. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 1414–1422, Marseille, France. European Language Resources Association.
Cite (Informal):
When Shallow is Good Enough: Automatic Assessment of Conceptual Text Complexity using Shallow Semantic Features (Stajner & Hulpuș, LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.177.pdf