Gated-Attention Readers for Text Comprehension

Bhuwan Dhingra, Hanxiao Liu, Zhilin Yang, William Cohen, Ruslan Salakhutdinov


Abstract
In this paper we study the problem of answering cloze-style questions over documents. Our model, the Gated-Attention (GA) Reader, integrates a multi-hop architecture with a novel attention mechanism, which is based on multiplicative interactions between the query embedding and the intermediate states of a recurrent neural network document reader. This enables the reader to build query-specific representations of tokens in the document for accurate answer selection. The GA Reader obtains state-of-the-art results on three benchmarks for this task–the CNN & Daily Mail news stories and the Who Did What dataset. The effectiveness of multiplicative interaction is demonstrated by an ablation study, and by comparing to alternative compositional operators for implementing the gated-attention.
Anthology ID:
P17-1168
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Editors:
Regina Barzilay, Min-Yen Kan
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1832–1846
Language:
URL:
https://aclanthology.org/P17-1168
DOI:
10.18653/v1/P17-1168
Bibkey:
Cite (ACL):
Bhuwan Dhingra, Hanxiao Liu, Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. 2017. Gated-Attention Readers for Text Comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1832–1846, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Gated-Attention Readers for Text Comprehension (Dhingra et al., ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-1168.pdf
Code
 bdhingra/ga-reader +  additional community code
Data
BookTestCBTCNN/Daily MailChildren's Book TestQUASARQUASAR-T