Turn-Level User Satisfaction Estimation in E-commerce Customer Service

Runze Liang, Ryuichi Takanobu, Feng-Lin Li, Ji Zhang, Haiqing Chen, Minlie Huang


Abstract
User satisfaction estimation in the dialogue-based customer service is critical not only for helping developers find the system defects, but also making it possible to get timely human intervention for dissatisfied customers. In this paper, we investigate the problem of user satisfaction estimation in E-commerce customer service. In order to apply the estimator to online services for timely human intervention, we need to estimate the satisfaction score at each turn. However, in actual scenario we can only collect the satisfaction labels for the whole dialogue sessions via user feedback. To this end, we formalize the turn-level satisfaction estimation as a reinforcement learning problem, in which the model can be optimized with only session-level satisfaction labels. We conduct experiments on the dataset collected from a commercial customer service system, and compare our model with the supervised learning models. Extensive experiments show that the proposed method outperforms all the baseline models.
Anthology ID:
2021.ecnlp-1.4
Volume:
Proceedings of the 4th Workshop on e-Commerce and NLP
Month:
August
Year:
2021
Address:
Online
Editors:
Shervin Malmasi, Surya Kallumadi, Nicola Ueffing, Oleg Rokhlenko, Eugene Agichtein, Ido Guy
Venue:
ECNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
26–32
Language:
URL:
https://aclanthology.org/2021.ecnlp-1.4
DOI:
10.18653/v1/2021.ecnlp-1.4
Bibkey:
Cite (ACL):
Runze Liang, Ryuichi Takanobu, Feng-Lin Li, Ji Zhang, Haiqing Chen, and Minlie Huang. 2021. Turn-Level User Satisfaction Estimation in E-commerce Customer Service. In Proceedings of the 4th Workshop on e-Commerce and NLP, pages 26–32, Online. Association for Computational Linguistics.
Cite (Informal):
Turn-Level User Satisfaction Estimation in E-commerce Customer Service (Liang et al., ECNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.ecnlp-1.4.pdf