Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training

Wangchunshu Zhou, Qifei Li, Chenle Li


Abstract
In this paper, we propose Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems to avoid generic responses and model dialogue history better. In contrast to standard adversarial training algorithms, IAT encourages the model to be sensitive to the perturbation in the dialogue history and therefore learning from perturbations. By giving higher rewards for responses whose output probability reduces more significantly when dialogue history is perturbed, the model is encouraged to generate more diverse and consistent responses. By penalizing the model when generating the same response given perturbed dialogue history, the model is forced to better capture dialogue history and generate more informative responses. Experimental results on two benchmark datasets show that our approach can better model dialogue history and generate more diverse and consistent responses. In addition, we point out a problem of the widely used maximum mutual information (MMI) based methods for improving the diversity of dialogue response generation models and demonstrate it empirically.
Anthology ID:
2021.acl-long.57
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Editors:
Chengqing Zong, Fei Xia, Wenjie Li, Roberto Navigli
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
694–703
Language:
URL:
https://aclanthology.org/2021.acl-long.57
DOI:
10.18653/v1/2021.acl-long.57
Bibkey:
Cite (ACL):
Wangchunshu Zhou, Qifei Li, and Chenle Li. 2021. Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 694–703, Online. Association for Computational Linguistics.
Cite (Informal):
Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training (Zhou et al., ACL-IJCNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-long.57.pdf
Video:
 https://aclanthology.org/2021.acl-long.57.mp4
Data
DailyDialogOpenSubtitles