Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues

Thibault Cordier, Tanguy Urvoy, Fabrice Lefèvre, Lina M. Rojas Barahona


Abstract
Reinforcement learning has been widely adopted to model dialogue managers in task-oriented dialogues. However, the user simulator provided by state-of-the-art dialogue frameworks are only rough approximations of human behaviour. The ability to learn from a small number of human interactions is hence crucial, especially on multi-domain and multi-task environments where the action space is large. We therefore propose to use structured policies to improve sample efficiency when learning on these kinds of environments. We also evaluate the impact of learning from human vs simulated experts. Among the different levels of structure that we tested, the graph neural networks (GNNs) show a remarkable superiority by reaching a success rate above 80% with only 50 dialogues when learning from simulated experts. They also show superiority when learning from human experts, although a performance drop was observed. We therefore suggest to concentrate future research efforts on bridging the gap between human data, simulators and automatic evaluators in dialogue frameworks.
Anthology ID:
2023.findings-eacl.32
Volume:
Findings of the Association for Computational Linguistics: EACL 2023
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
432–441
Language:
URL:
https://aclanthology.org/2023.findings-eacl.32
DOI:
10.18653/v1/2023.findings-eacl.32
Bibkey:
Cite (ACL):
Thibault Cordier, Tanguy Urvoy, Fabrice Lefèvre, and Lina M. Rojas Barahona. 2023. Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues. In Findings of the Association for Computational Linguistics: EACL 2023, pages 432–441, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues (Cordier et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-eacl.32.pdf
Video:
 https://aclanthology.org/2023.findings-eacl.32.mp4