Diversity driven attention model for query-based abstractive summarization

Preksha Nema, Mitesh M. Khapra, Anirban Laha, Balaraman Ravindran


Abstract
Abstractive summarization aims to generate a shorter version of the document covering all the salient points in a compact and coherent fashion. On the other hand, query-based summarization highlights those points that are relevant in the context of a given query. The encode-attend-decode paradigm has achieved notable success in machine translation, extractive summarization, dialog systems, etc. But it suffers from the drawback of generation of repeated phrases. In this work we propose a model for the query-based summarization task based on the encode-attend-decode paradigm with two key additions (i) a query attention model (in addition to document attention model) which learns to focus on different portions of the query at different time steps (instead of using a static representation for the query) and (ii) a new diversity based attention model which aims to alleviate the problem of repeating phrases in the summary. In order to enable the testing of this model we introduce a new query-based summarization dataset building on debatepedia. Our experiments show that with these two additions the proposed model clearly outperforms vanilla encode-attend-decode models with a gain of 28% (absolute) in ROUGE-L scores.
Anthology ID:
P17-1098
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1063–1072
Language:
URL:
https://aclanthology.org/P17-1098
DOI:
10.18653/v1/P17-1098
Bibkey:
Cite (ACL):
Preksha Nema, Mitesh M. Khapra, Anirban Laha, and Balaraman Ravindran. 2017. Diversity driven attention model for query-based abstractive summarization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1063–1072, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Diversity driven attention model for query-based abstractive summarization (Nema et al., ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-1098.pdf
Video:
 https://vimeo.com/234956203
Code
 PrekshaNema25/DiverstiyBasedAttentionMechanism +  additional community code