DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation

Xinyu Hua, Ashwin Sreevatsa, Lu Wang


Abstract
We study the task of long-form opinion text generation, which faces at least two distinct challenges. First, existing neural generation models fall short of coherence, thus requiring efficient content planning. Second, diverse types of information are needed to guide the generator to cover both subjective and objective content. To this end, we propose DYPLOC, a generation framework that conducts dynamic planning of content while generating the output based on a novel design of mixed language models. To enrich the generation with diverse content, we further propose to use large pre-trained models to predict relevant concepts and to generate claims. We experiment with two challenging tasks on newly collected datasets: (1) argument generation with Reddit ChangeMyView, and (2) writing articles using New York Times’ Opinion section. Automatic evaluation shows that our model significantly outperforms competitive comparisons. Human judges further confirm that our generations are more coherent with richer content.
Anthology ID:
2021.acl-long.501
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Editors:
Chengqing Zong, Fei Xia, Wenjie Li, Roberto Navigli
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6408–6423
Language:
URL:
https://aclanthology.org/2021.acl-long.501
DOI:
10.18653/v1/2021.acl-long.501
Bibkey:
Cite (ACL):
Xinyu Hua, Ashwin Sreevatsa, and Lu Wang. 2021. DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6408–6423, Online. Association for Computational Linguistics.
Cite (Informal):
DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation (Hua et al., ACL-IJCNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-long.501.pdf
Video:
 https://aclanthology.org/2021.acl-long.501.mp4
Data
ConceptNetNew York Times Annotated Corpus