Broad Context Language Modeling as Reading Comprehension

Zewei Chu, Hai Wang, Kevin Gimpel, David McAllester


Abstract
Progress in text understanding has been driven by large datasets that test particular capabilities, like recent datasets for reading comprehension (Hermann et al., 2015). We focus here on the LAMBADA dataset (Paperno et al., 2016), a word prediction task requiring broader context than the immediate sentence. We view LAMBADA as a reading comprehension problem and apply comprehension models based on neural networks. Though these models are constrained to choose a word from the context, they improve the state of the art on LAMBADA from 7.3% to 49%. We analyze 100 instances, finding that neural network readers perform well in cases that involve selecting a name from the context based on dialogue or discourse cues but struggle when coreference resolution or external knowledge is needed.
Anthology ID:
E17-2009
Volume:
Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers
Month:
April
Year:
2017
Address:
Valencia, Spain
Editors:
Mirella Lapata, Phil Blunsom, Alexander Koller
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
52–57
Language:
URL:
https://aclanthology.org/E17-2009
DOI:
Bibkey:
Cite (ACL):
Zewei Chu, Hai Wang, Kevin Gimpel, and David McAllester. 2017. Broad Context Language Modeling as Reading Comprehension. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 52–57, Valencia, Spain. Association for Computational Linguistics.
Cite (Informal):
Broad Context Language Modeling as Reading Comprehension (Chu et al., EACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/E17-2009.pdf
Data
BookCorpusLAMBADA