Improved Document Modelling with a Neural Discourse Parser

Fajri Koto, Jey Han Lau, Timothy Baldwin


Abstract
Despite the success of attention-based neural models for natural language generation and classification tasks, they are unable to capture the discourse structure of larger documents. We hypothesize that explicit discourse representations have utility for NLP tasks over longer documents or document sequences, which sequence-to-sequence models are unable to capture. For abstractive summarization, for instance, conventional neural models simply match source documents and the summary in a latent space without explicit representation of text structure or relations. In this paper, we propose to use neural discourse representations obtained from a rhetorical structure theory (RST) parser to enhance document representations. Specifically, document representations are generated for discourse spans, known as the elementary discourse units (EDUs). We empirically investigate the benefit of the proposed approach on two different tasks: abstractive summarization and popularity prediction of online petitions. We find that the proposed approach leads to substantial improvements in all cases.
Anthology ID:
U19-1010
Volume:
Proceedings of the 17th Annual Workshop of the Australasian Language Technology Association
Month:
4--6 December
Year:
2019
Address:
Sydney, Australia
Editors:
Meladel Mistica, Massimo Piccardi, Andrew MacKinlay
Venue:
ALTA
SIG:
Publisher:
Australasian Language Technology Association
Note:
Pages:
67–76
Language:
URL:
https://aclanthology.org/U19-1010
DOI:
Bibkey:
Cite (ACL):
Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2019. Improved Document Modelling with a Neural Discourse Parser. In Proceedings of the 17th Annual Workshop of the Australasian Language Technology Association, pages 67–76, Sydney, Australia. Australasian Language Technology Association.
Cite (Informal):
Improved Document Modelling with a Neural Discourse Parser (Koto et al., ALTA 2019)
Copy Citation:
PDF:
https://aclanthology.org/U19-1010.pdf
Code
 fajri91/RSTExtractor