Multi-Action Dialog Policy Learning with Interactive Human Teaching

Megha Jhunjhunwala, Caleb Bryant, Pararth Shah


Abstract
We present a framework for improving task-oriented dialog systems through online interactive teaching with human trainers. A dialog policy trained with imitation learning on a limited corpus may not generalize well to novel dialog flows often uncovered in live interactions. This issue is magnified in multi-action dialog policies which have a more expressive action space. In our approach, a pre-trained dialog policy model interacts with human trainers, and at each turn the trainers choose the best output among N-best multi-action outputs. We present a novel multi-domain, multi-action dialog policy architecture trained on MultiWOZ, and show that small amounts of online supervision can lead to significant improvement in model performance. We also present transfer learning results which show that interactive learning in one domain improves policy model performance in related domains.
Anthology ID:
2020.sigdial-1.36
Volume:
Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Month:
July
Year:
2020
Address:
1st virtual meeting
Editors:
Olivier Pietquin, Smaranda Muresan, Vivian Chen, Casey Kennington, David Vandyke, Nina Dethlefs, Koji Inoue, Erik Ekstedt, Stefan Ultes
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
290–296
Language:
URL:
https://aclanthology.org/2020.sigdial-1.36
DOI:
10.18653/v1/2020.sigdial-1.36
Bibkey:
Cite (ACL):
Megha Jhunjhunwala, Caleb Bryant, and Pararth Shah. 2020. Multi-Action Dialog Policy Learning with Interactive Human Teaching. In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 290–296, 1st virtual meeting. Association for Computational Linguistics.
Cite (Informal):
Multi-Action Dialog Policy Learning with Interactive Human Teaching (Jhunjhunwala et al., SIGDIAL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.sigdial-1.36.pdf
Video:
 https://youtube.com/watch?v=_0WiUcv_KNI