RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generators

Rilwan Adewoyin, Ritabrata Dutta, Yulan He


Abstract
In this paper, we study the task of improving the cohesion and coherence of long-form text generated by language models. To this end, we propose RSTGen, a framework that utilises Rhetorical Structure Theory (RST), a classical language theory, to control the discourse structure, semantics and topics of generated text. Firstly, we demonstrate our model’s ability to control structural discourse and semantic features of generated text in open generation evaluation. Then we experiment on the two challenging long-form text tasks of argument generation and story generation. Evaluation using automated metrics and a metric with high correlation to human evaluation, shows that our model performs competitively against existing models, while offering significantly more controls over generated text than alternative methods.
Anthology ID:
2022.naacl-main.133
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1822–1835
Language:
URL:
https://aclanthology.org/2022.naacl-main.133
DOI:
10.18653/v1/2022.naacl-main.133
Bibkey:
Cite (ACL):
Rilwan Adewoyin, Ritabrata Dutta, and Yulan He. 2022. RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generators. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1822–1835, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generators (Adewoyin et al., NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-main.133.pdf
Video:
 https://aclanthology.org/2022.naacl-main.133.mp4
Data
WritingPrompts