Improving Interaction Quality Estimation with BiLSTMs and the Impact on Dialogue Policy Learning

Stefan Ultes


Abstract
Learning suitable and well-performing dialogue behaviour in statistical spoken dialogue systems has been in the focus of research for many years. While most work which is based on reinforcement learning employs an objective measure like task success for modelling the reward signal, we use a reward based on user satisfaction estimation. We propose a novel estimator and show that it outperforms all previous estimators while learning temporal dependencies implicitly. Furthermore, we apply this novel user satisfaction estimation model live in simulated experiments where the satisfaction estimation model is trained on one domain and applied in many other domains which cover a similar task. We show that applying this model results in higher estimated satisfaction, similar task success rates and a higher robustness to noise.
Anthology ID:
W19-5902
Volume:
Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue
Month:
September
Year:
2019
Address:
Stockholm, Sweden
Editors:
Satoshi Nakamura, Milica Gasic, Ingrid Zukerman, Gabriel Skantze, Mikio Nakano, Alexandros Papangelis, Stefan Ultes, Koichiro Yoshino
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
11–20
Language:
URL:
https://aclanthology.org/W19-5902
DOI:
10.18653/v1/W19-5902
Bibkey:
Cite (ACL):
Stefan Ultes. 2019. Improving Interaction Quality Estimation with BiLSTMs and the Impact on Dialogue Policy Learning. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, pages 11–20, Stockholm, Sweden. Association for Computational Linguistics.
Cite (Informal):
Improving Interaction Quality Estimation with BiLSTMs and the Impact on Dialogue Policy Learning (Ultes, SIGDIAL 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-5902.pdf