Predicting Users’ Negative Feedbacks in Multi-Turn Human-Computer Dialogues

Xin Wang, Jianan Wang, Yuanchao Liu, Xiaolong Wang, Zhuoran Wang, Baoxun Wang


Abstract
User experience is essential for human-computer dialogue systems. However, it is impractical to ask users to provide explicit feedbacks when the agents’ responses displease them. Therefore, in this paper, we explore to predict users’ imminent dissatisfactions caused by intelligent agents by analysing the existing utterances in the dialogue sessions. To our knowledge, this is the first work focusing on this task. Several possible factors that trigger negative emotions are modelled. A relation sequence model (RSM) is proposed to encode the sequence of appropriateness of current response with respect to the earlier utterances. The experimental results show that the proposed structure is effective in modelling emotional risk (possibility of negative feedback) than existing conversation modelling approaches. Besides, strategies of obtaining distance supervision data for pre-training are also discussed in this work. Balanced sampling with respect to the last response in the distance supervision data are shown to be reliable for data augmentation.
Anthology ID:
I17-1072
Volume:
Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
November
Year:
2017
Address:
Taipei, Taiwan
Editors:
Greg Kondrak, Taro Watanabe
Venue:
IJCNLP
SIG:
Publisher:
Asian Federation of Natural Language Processing
Note:
Pages:
713–722
Language:
URL:
https://aclanthology.org/I17-1072
DOI:
Bibkey:
Cite (ACL):
Xin Wang, Jianan Wang, Yuanchao Liu, Xiaolong Wang, Zhuoran Wang, and Baoxun Wang. 2017. Predicting Users’ Negative Feedbacks in Multi-Turn Human-Computer Dialogues. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 713–722, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Cite (Informal):
Predicting Users’ Negative Feedbacks in Multi-Turn Human-Computer Dialogues (Wang et al., IJCNLP 2017)
Copy Citation:
PDF:
https://aclanthology.org/I17-1072.pdf