Memory-bounded Neural Incremental Parsing for Psycholinguistic Prediction

Lifeng Jin, William Schuler


Abstract
Syntactic surprisal has been shown to have an effect on human sentence processing, and can be predicted from prefix probabilities of generative incremental parsers. Recent state-of-the-art incremental generative neural parsers are able to produce accurate parses and surprisal values but have unbounded stack memory, which may be used by the neural parser to maintain explicit in-order representations of all previously parsed words, inconsistent with results of human memory experiments. In contrast, humans seem to have a bounded working memory, demonstrated by inhibited performance on word recall in multi-clause sentences (Bransford and Franks, 1971), and on center-embedded sentences (Miller and Isard,1964). Bounded statistical parsers exist, but are less accurate than neural parsers in predict-ing reading times. This paper describes a neural incremental generative parser that is able to provide accurate surprisal estimates and can be constrained to use a bounded stack. Results show that the accuracy gains of neural parsers can be reliably extended to psycholinguistic modeling without risk of distortion due to un-bounded working memory.
Anthology ID:
2020.iwpt-1.6
Volume:
Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies
Month:
July
Year:
2020
Address:
Online
Editors:
Gosse Bouma, Yuji Matsumoto, Stephan Oepen, Kenji Sagae, Djamé Seddah, Weiwei Sun, Anders Søgaard, Reut Tsarfaty, Dan Zeman
Venue:
IWPT
SIG:
SIGPARSE
Publisher:
Association for Computational Linguistics
Note:
Pages:
48–61
Language:
URL:
https://aclanthology.org/2020.iwpt-1.6
DOI:
10.18653/v1/2020.iwpt-1.6
Bibkey:
Cite (ACL):
Lifeng Jin and William Schuler. 2020. Memory-bounded Neural Incremental Parsing for Psycholinguistic Prediction. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies, pages 48–61, Online. Association for Computational Linguistics.
Cite (Informal):
Memory-bounded Neural Incremental Parsing for Psycholinguistic Prediction (Jin & Schuler, IWPT 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.iwpt-1.6.pdf
Video:
 http://slideslive.com/38929673