Discourse-Aware Neural Extractive Text Summarization

Jiacheng Xu, Zhe Gan, Yu Cheng, Jingjing Liu


Abstract
Recently BERT has been adopted for document encoding in state-of-the-art text summarization models. However, sentence-based extractive models often result in redundant or uninformative phrases in the extracted summaries. Also, long-range dependencies throughout a document are not well captured by BERT, which is pre-trained on sentence pairs instead of documents. To address these issues, we present a discourse-aware neural summarization model - DiscoBert. DiscoBert extracts sub-sentential discourse units (instead of sentences) as candidates for extractive selection on a finer granularity. To capture the long-range dependencies among discourse units, structural discourse graphs are constructed based on RST trees and coreference mentions, encoded with Graph Convolutional Networks. Experiments show that the proposed model outperforms state-of-the-art methods by a significant margin on popular summarization benchmarks compared to other BERT-base models.
Anthology ID:
2020.acl-main.451
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5021–5031
Language:
URL:
https://aclanthology.org/2020.acl-main.451
DOI:
10.18653/v1/2020.acl-main.451
Bibkey:
Cite (ACL):
Jiacheng Xu, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. Discourse-Aware Neural Extractive Text Summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5021–5031, Online. Association for Computational Linguistics.
Cite (Informal):
Discourse-Aware Neural Extractive Text Summarization (Xu et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.451.pdf
Video:
 http://slideslive.com/38929202
Code
 jiacheng-xu/DiscoBERT
Data
New York Times Annotated Corpus