Towards Optimizing Pre-trained Language Model Ensemble Learning for Task-oriented Dialogue System

Zhiyuan Zhu, Yusheng Liao, Zhe Chen, Yu Wang, Yunfeng Guan


Abstract
Task-oriented dialogue systems that employ external knowledge to generate informative responses have become an important field of research. This paper outlines our contribution to Track 5 of the Eleventh Dialog System Technology Challenge (DSTC11), which focuses on constructing high-performing, subjective knowledge-enriched task-oriented dialogue systems. Specifically, we investigate the complementarity of various language models to tackle the diverse knowledge selection task that involves multiple external sources. Based on this investigation, we propose pre- and post-generation model ensemble approaches to mitigate potential biases inherent in using a single model for the knowledge selection task. Finally, we utilize the consensus decoding approach to combine fine-tuned ensemble models and improve the performance of the generation system. Our system ranked 1st in human evaluation, even outperforming human annotation.
Anthology ID:
2023.dstc-1.17
Volume:
Proceedings of The Eleventh Dialog System Technology Challenge
Month:
September
Year:
2023
Address:
Prague, Czech Republic
Editors:
Yun-Nung Chen, Paul Crook, Michel Galley, Sarik Ghazarian, Chulaka Gunasekara, Raghav Gupta, Behnam Hedayatnia, Satwik Kottur, Seungwhan Moon, Chen Zhang
Venues:
DSTC | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
144–149
Language:
URL:
https://aclanthology.org/2023.dstc-1.17
DOI:
Bibkey:
Cite (ACL):
Zhiyuan Zhu, Yusheng Liao, Zhe Chen, Yu Wang, and Yunfeng Guan. 2023. Towards Optimizing Pre-trained Language Model Ensemble Learning for Task-oriented Dialogue System. In Proceedings of The Eleventh Dialog System Technology Challenge, pages 144–149, Prague, Czech Republic. Association for Computational Linguistics.
Cite (Informal):
Towards Optimizing Pre-trained Language Model Ensemble Learning for Task-oriented Dialogue System (Zhu et al., DSTC-WS 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.dstc-1.17.pdf