StructuralCost: A controlled reading time dataset for modeling human sentence processing difficulty

Nina Nusbaumer, Iria de-Dios-Flores, Corentin Bel, Christophe Pallier, Guillaume Wisniewski, Benoit Crabbé


Abstract
We introduce StructuralCost, a self-paced reading dataset of 475 participants and 40,800 observations isolating the processing cost of long-distance subject-verb dependency resolution. We replicate a low-powered psycholinguistic finding at NLP scale, namely that human reading times at the main verb increase with dependency length, driven by syntactic embedding beyond linear distance. Different language models (spanning n-gram models, SSMs, and transformers) partially mirror this graded difficulty profile, yet underestimate the integration cost humans incur, with a gap that persists across architectures and model sizes. This suggests these models capture the predictive component of human processing but not the full integration cost that working memory imposes. StructuralCost provides data needed to drive progress toward evaluating the cognitive plausibility of language models.
Anthology ID:
2026.findings-emnlp.1360
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
28874–28888
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1360/
DOI:
Bibkey:
Cite (ACL):
Nina Nusbaumer, Iria de-Dios-Flores, Corentin Bel, Christophe Pallier, Guillaume Wisniewski, and Benoit Crabbé. 2026. StructuralCost: A controlled reading time dataset for modeling human sentence processing difficulty. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 28874–28888, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
StructuralCost: A controlled reading time dataset for modeling human sentence processing difficulty (Nusbaumer et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1360.pdf