Combining Cognitive Modeling and Reinforcement Learning for Clarification in Dialogue

Baber Khalid, Malihe Alikhani, Matthew Stone


Abstract
In many domains, dialogue systems need to work collaboratively with users to successfully reconstruct the meaning the user had in mind. In this paper, we show how cognitive models of users’ communicative strategies can be leveraged in a reinforcement learning approach to dialogue planning to enable interactive systems to give targeted, effective feedback about the system’s understanding. We describe a prototype system that collaborates on reference tasks that distinguish arbitrarily varying color patches from similar distractors, and use experiments with crowd workers and analyses of our learned policies to document that our approach leads to context-sensitive clarification strategies that focus on key missing information, elicit correct answers that the system understands, and contribute to increasing dialogue success.
Anthology ID:
2020.coling-main.391
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Donia Scott, Nuria Bel, Chengqing Zong
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
4417–4428
Language:
URL:
https://aclanthology.org/2020.coling-main.391
DOI:
10.18653/v1/2020.coling-main.391
Bibkey:
Cite (ACL):
Baber Khalid, Malihe Alikhani, and Matthew Stone. 2020. Combining Cognitive Modeling and Reinforcement Learning for Clarification in Dialogue. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4417–4428, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
Combining Cognitive Modeling and Reinforcement Learning for Clarification in Dialogue (Khalid et al., COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-main.391.pdf