Patterns of Text Readability in Human and Predicted Eye Movements

Nora Hollenstein, Itziar Gonzalez-Dios, Lisa Beinborn, Lena Jäger


Abstract
It has been shown that multilingual transformer models are able to predict human reading behavior when fine-tuned on small amounts of eye tracking data. As the cumulated prediction results do not provide insights into the linguistic cues that the model acquires to predict reading behavior, we conduct a deeper analysis of the predictions from the perspective of readability. We try to disentangle the three-fold relationship between human eye movements, the capability of language models to predict these eye movement patterns, and sentence-level readability measures for English. We compare a range of model configurations to multiple baselines. We show that the models exhibit difficulties with function words and that pre-training only provides limited advantages for linguistic generalization.
Anthology ID:
2022.cogalex-1.1
Volume:
Proceedings of the Workshop on Cognitive Aspects of the Lexicon
Month:
November
Year:
2022
Address:
Taipei, Taiwan
Editors:
Michael Zock, Emmanuele Chersoni, Yu-Yin Hsu, Enrico Santus
Venue:
CogALex
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–15
Language:
URL:
https://aclanthology.org/2022.cogalex-1.1
DOI:
Bibkey:
Cite (ACL):
Nora Hollenstein, Itziar Gonzalez-Dios, Lisa Beinborn, and Lena Jäger. 2022. Patterns of Text Readability in Human and Predicted Eye Movements. In Proceedings of the Workshop on Cognitive Aspects of the Lexicon, pages 1–15, Taipei, Taiwan. Association for Computational Linguistics.
Cite (Informal):
Patterns of Text Readability in Human and Predicted Eye Movements (Hollenstein et al., CogALex 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.cogalex-1.1.pdf