Posterior Control of Blackbox Generation

Xiang Lisa Li, Alexander Rush


Abstract
Text generation often requires high-precision output that obeys task-specific rules. This fine-grained control is difficult to enforce with off-the-shelf deep learning models. In this work, we consider augmenting neural generation models with discrete control states learned through a structured latent-variable approach. Under this formulation, task-specific knowledge can be encoded through a range of rich, posterior constraints that are effectively trained into the model. This approach allows users to ground internal model decisions based on prior knowledge, without sacrificing the representational power of neural generative models. Experiments consider applications of this approach for text generation. We find that this method improves over standard benchmarks, while also providing fine-grained control.
Anthology ID:
2020.acl-main.243
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2731–2743
Language:
URL:
https://aclanthology.org/2020.acl-main.243
DOI:
10.18653/v1/2020.acl-main.243
Bibkey:
Cite (ACL):
Xiang Lisa Li and Alexander Rush. 2020. Posterior Control of Blackbox Generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2731–2743, Online. Association for Computational Linguistics.
Cite (Informal):
Posterior Control of Blackbox Generation (Li & Rush, ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.243.pdf
Video:
 http://slideslive.com/38929245
Code
 XiangLi1999/PosteriorControl-NLG +  additional community code
Data
WikiBio