BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data

Haoyu Song, Yan Wang, Kaiyan Zhang, Wei-Nan Zhang, Ting Liu


Abstract
Maintaining a consistent persona is essential for dialogue agents. Although tremendous advancements have been brought, the limited-scale of annotated personalized dialogue datasets is still a barrier towards training robust and consistent persona-based dialogue models. This work shows how this challenge can be addressed by disentangling persona-based dialogue generation into two sub-tasks with a novel BERT-over-BERT (BoB) model. Specifically, the model consists of a BERT-based encoder and two BERT-based decoders, where one decoder is for response generation, and another is for consistency understanding. In particular, to learn the ability of consistency understanding from large-scale non-dialogue inference data, we train the second decoder in an unlikelihood manner. Under different limited data settings, both automatic and human evaluations demonstrate that the proposed model outperforms strong baselines in response quality and persona consistency.
Anthology ID:
2021.acl-long.14
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
167–177
Language:
URL:
https://aclanthology.org/2021.acl-long.14
DOI:
10.18653/v1/2021.acl-long.14
Bibkey:
Cite (ACL):
Haoyu Song, Yan Wang, Kaiyan Zhang, Wei-Nan Zhang, and Ting Liu. 2021. BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 167–177, Online. Association for Computational Linguistics.
Cite (Informal):
BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data (Song et al., ACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-long.14.pdf
Video:
 https://aclanthology.org/2021.acl-long.14.mp4
Code
 songhaoyu/BoB
Data
ConvAI2MultiNLIPersonalDialog