CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns

Peter Vickers, Rosa Wainwright, Harish Tayyar Madabushi, Aline Villavicencio


Abstract
The CogNLP-Sheffield submissions to the CMCL 2021 Shared Task examine the value of a variety of cognitively and linguistically inspired features for predicting eye tracking patterns, as both standalone model inputs and as supplements to contextual word embeddings (XLNet). Surprisingly, the smaller pre-trained model (XLNet-base) outperforms the larger (XLNet-large), and despite evidence that multi-word expressions (MWEs) provide cognitive processing advantages, MWE features provide little benefit to either model.
Anthology ID:
2021.cmcl-1.16
Volume:
Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
Month:
June
Year:
2021
Address:
Online
Editors:
Emmanuele Chersoni, Nora Hollenstein, Cassandra Jacobs, Yohei Oseki, Laurent Prévot, Enrico Santus
Venue:
CMCL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
125–133
Language:
URL:
https://aclanthology.org/2021.cmcl-1.16
DOI:
10.18653/v1/2021.cmcl-1.16
Bibkey:
Cite (ACL):
Peter Vickers, Rosa Wainwright, Harish Tayyar Madabushi, and Aline Villavicencio. 2021. CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 125–133, Online. Association for Computational Linguistics.
Cite (Informal):
CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns (Vickers et al., CMCL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.cmcl-1.16.pdf
Data
WikiText-103WikiText-2