Informed Sampling for Diversity in Concept-to-Text NLG

Giulio Zhou, Gerasimos Lampouras


Abstract
Deep-learning models for language generation tasks tend to produce repetitive output. Various methods have been proposed to encourage lexical diversity during decoding, but this often comes at a cost to the perceived fluency and adequacy of the output. In this work, we propose to ameliorate this cost by using an Imitation Learning approach to explore the level of diversity that a language generation model can reliably produce. Specifically, we augment the decoding process with a meta-classifier trained to distinguish which words at any given timestep will lead to high-quality output. We focus our experiments on concept-to-text generation where models are sensitive to the inclusion of irrelevant words due to the strict relation between input and output. Our analysis shows that previous methods for diversity underperform in this setting, while human evaluation suggests that our proposed method achieves a high level of diversity with minimal effect on the output’s fluency and adequacy.
Anthology ID:
2021.findings-emnlp.213
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2021
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
2494–2509
Language:
URL:
https://aclanthology.org/2021.findings-emnlp.213
DOI:
10.18653/v1/2021.findings-emnlp.213
Bibkey:
Cite (ACL):
Giulio Zhou and Gerasimos Lampouras. 2021. Informed Sampling for Diversity in Concept-to-Text NLG. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2494–2509, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Informed Sampling for Diversity in Concept-to-Text NLG (Zhou & Lampouras, Findings 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.findings-emnlp.213.pdf
Video:
 https://aclanthology.org/2021.findings-emnlp.213.mp4
Data
MultiWOZ