An Empirical Bayes Framework for Open-Domain Dialogue Generation

Jing Yang Lee, Kong Aik Lee, Woon Seng Gan


Abstract
To engage human users in meaningful conversation, open-domain dialogue agents are required to generate diverse and contextually coherent dialogue. Despite recent advancements, which can be attributed to the usage of pretrained language models, the generation of diverse and coherent dialogue remains an open research problem. A popular approach to address this issue involves the adaptation of variational frameworks. However, while these approaches successfully improve diversity, they tend to compromise on contextual coherence. Hence, we propose the Bayesian Open-domain Dialogue with Empirical Bayes (BODEB) framework, an empirical bayes framework for constructing an Bayesian open-domain dialogue agent by leveraging pretrained parameters to inform the prior and posterior parameter distributions. Empirical results show that BODEB achieves better results in terms of both diversity and coherence compared to variational frameworks.
Anthology ID:
2023.gem-1.17
Volume:
Proceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM)
Month:
December
Year:
2023
Address:
Singapore
Editors:
Sebastian Gehrmann, Alex Wang, João Sedoc, Elizabeth Clark, Kaustubh Dhole, Khyathi Raghavi Chandu, Enrico Santus, Hooman Sedghamiz
Venues:
GEM | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
192–204
Language:
URL:
https://aclanthology.org/2023.gem-1.17
DOI:
Bibkey:
Cite (ACL):
Jing Yang Lee, Kong Aik Lee, and Woon Seng Gan. 2023. An Empirical Bayes Framework for Open-Domain Dialogue Generation. In Proceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), pages 192–204, Singapore. Association for Computational Linguistics.
Cite (Informal):
An Empirical Bayes Framework for Open-Domain Dialogue Generation (Lee et al., GEM-WS 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.gem-1.17.pdf