Incremental Learning from Scratch for Task-Oriented Dialogue Systems

Weikang Wang, Jiajun Zhang, Qian Li, Mei-Yuh Hwang, Chengqing Zong, Zhifei Li


Abstract
Clarifying user needs is essential for existing task-oriented dialogue systems. However, in real-world applications, developers can never guarantee that all possible user demands are taken into account in the design phase. Consequently, existing systems will break down when encountering unconsidered user needs. To address this problem, we propose a novel incremental learning framework to design task-oriented dialogue systems, or for short Incremental Dialogue System (IDS), without pre-defining the exhaustive list of user needs. Specifically, we introduce an uncertainty estimation module to evaluate the confidence of giving correct responses. If there is high confidence, IDS will provide responses to users. Otherwise, humans will be involved in the dialogue process, and IDS can learn from human intervention through an online learning module. To evaluate our method, we propose a new dataset which simulates unanticipated user needs in the deployment stage. Experiments show that IDS is robust to unconsidered user actions, and can update itself online by smartly selecting only the most effective training data, and hence attains better performance with less annotation cost.
Anthology ID:
P19-1361
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3710–3720
Language:
URL:
https://aclanthology.org/P19-1361
DOI:
10.18653/v1/P19-1361
Bibkey:
Cite (ACL):
Weikang Wang, Jiajun Zhang, Qian Li, Mei-Yuh Hwang, Chengqing Zong, and Zhifei Li. 2019. Incremental Learning from Scratch for Task-Oriented Dialogue Systems. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3710–3720, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Incremental Learning from Scratch for Task-Oriented Dialogue Systems (Wang et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1361.pdf
Supplementary:
 P19-1361.Supplementary.zip
Code
 Leechikara/Incremental-Dialogue-System
Data
Incremental Dialog Dataset