Prompting, Retrieval, Training: An exploration of different approaches for task-oriented dialogue generation

Gonçalo Raposo, Luisa Coheur, Bruno Martins


Abstract
Task-oriented dialogue systems need to generate appropriate responses to help fulfill users’ requests. This paper explores different strategies, namely prompting, retrieval, and fine-tuning, for task-oriented dialogue generation. Through a systematic evaluation, we aim to provide valuable insights and guidelines for researchers and practitioners working on developing efficient and effective dialogue systems for real-world applications. Evaluation is performed on the MultiWOZ and Taskmaster-2 datasets, and we test various versions of FLAN-T5, GPT-3.5, and GPT-4 models. Costs associated with running these models are analyzed, and dialogue evaluation is briefly discussed. Our findings suggest that when testing data differs from the training data, fine-tuning may decrease performance, favoring a combination of a more general language model and a prompting mechanism based on retrieved examples.
Anthology ID:
2023.sigdial-1.37
Volume:
Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Month:
September
Year:
2023
Address:
Prague, Czechia
Editors:
Svetlana Stoyanchev, Shafiq Joty, David Schlangen, Ondrej Dusek, Casey Kennington, Malihe Alikhani
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
400–412
Language:
URL:
https://aclanthology.org/2023.sigdial-1.37
DOI:
10.18653/v1/2023.sigdial-1.37
Bibkey:
Cite (ACL):
Gonçalo Raposo, Luisa Coheur, and Bruno Martins. 2023. Prompting, Retrieval, Training: An exploration of different approaches for task-oriented dialogue generation. In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 400–412, Prague, Czechia. Association for Computational Linguistics.
Cite (Informal):
Prompting, Retrieval, Training: An exploration of different approaches for task-oriented dialogue generation (Raposo et al., SIGDIAL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.sigdial-1.37.pdf