Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization

Dongqi Pu, Yifan Wang, Vera Demberg


Abstract
For text summarization, the role of discourse structure is pivotal in discerning the core content of a text. Regrettably, prior studies on incorporating Rhetorical Structure Theory (RST) into transformer-based summarization models only consider the nuclearity annotation, thereby overlooking the variety of discourse relation types. This paper introduces the ‘RSTformer’, a novel summarization model that comprehensively incorporates both the types and uncertainty of rhetorical relations. Our RST-attention mechanism, rooted in document-level rhetorical structure, is an extension of the recently devised Longformer framework. Through rigorous evaluation, the model proposed herein exhibits significant superiority over state-of-the-art models, as evidenced by its notable performance on several automatic metrics and human evaluation.
Anthology ID:
2023.acl-long.306
Volume:
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5574–5590
Language:
URL:
https://aclanthology.org/2023.acl-long.306
DOI:
10.18653/v1/2023.acl-long.306
Bibkey:
Cite (ACL):
Dongqi Pu, Yifan Wang, and Vera Demberg. 2023. Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5574–5590, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization (Pu et al., ACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.acl-long.306.pdf
Video:
 https://aclanthology.org/2023.acl-long.306.mp4