EM Pre-training for Multi-party Dialogue Response Generation

Yiyang Li, Hai Zhao


Abstract
Dialogue response generation requires an agent to generate a response according to the current dialogue history, in terms of which two-party dialogues have been well studied, but leaving a great gap for multi-party dialogues at the same time. Different from two-party dialogues where each response is a direct reply to its previous utterance, the addressee of a response utterance should be specified before it is generated in the multi-party scenario. Thanks to the huge amount of two-party conversational data, various pre-trained language models for two-party dialogue response generation have been proposed. However, due to the lack of annotated addressee labels in multi-party dialogue datasets, it is hard to use them to pre-train a response generation model for multi-party dialogues. To tackle this obstacle, we propose an Expectation-Maximization (EM) approach that iteratively performs the expectation steps to generate addressee labels, and the maximization steps to optimize a response generation model. Theoretical analyses and extensive experiments have justified the feasibility and effectiveness of our proposed method. The official implementation of this paper is available at https://github.com/EricLee8/MPDRG.
Anthology ID:
2023.acl-long.7
Volume:
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
92–103
Language:
URL:
https://aclanthology.org/2023.acl-long.7
DOI:
10.18653/v1/2023.acl-long.7
Bibkey:
Cite (ACL):
Yiyang Li and Hai Zhao. 2023. EM Pre-training for Multi-party Dialogue Response Generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 92–103, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
EM Pre-training for Multi-party Dialogue Response Generation (Li & Zhao, ACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.acl-long.7.pdf
Video:
 https://aclanthology.org/2023.acl-long.7.mp4