Agree to Disagree: Exploring Subjectivity in Lexical Complexity

Matthew Shardlow


Abstract
Subjective factors affect our familiarity with different words. Our education, mother tongue, dialect or social group all contribute to the words we know and understand. When asking people to mark words they understand some words are unanimously agreed to be complex, whereas other annotators universally disagree on the complexity of other words. In this work, we seek to expose this phenomenon and investigate the factors affecting whether a word is likely to be subjective, or not. We investigate two recent word complexity datasets from shared tasks. We demonstrate that subjectivity is present and describable in both datasets. Further we show results of modelling and predicting the subjectivity of the complexity annotations in the most recent dataset, attaining an F1-score of 0.714.
Anthology ID:
2022.readi-1.2
Volume:
Proceedings of the 2nd Workshop on Tools and Resources to Empower People with REAding DIfficulties (READI) within the 13th Language Resources and Evaluation Conference
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Rodrigo Wilkens, David Alfter, Rémi Cardon, Núria Gala
Venue:
READI
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
9–16
Language:
URL:
https://aclanthology.org/2022.readi-1.2
DOI:
Bibkey:
Cite (ACL):
Matthew Shardlow. 2022. Agree to Disagree: Exploring Subjectivity in Lexical Complexity. In Proceedings of the 2nd Workshop on Tools and Resources to Empower People with REAding DIfficulties (READI) within the 13th Language Resources and Evaluation Conference, pages 9–16, Marseille, France. European Language Resources Association.
Cite (Informal):
Agree to Disagree: Exploring Subjectivity in Lexical Complexity (Shardlow, READI 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.readi-1.2.pdf
Code
 mmu-tdmlab/lcp_subjectivity