A Situated Dialogue System for Learning Structural Concepts in Blocks World

Ian Perera, James Allen, Choh Man Teng, Lucian Galescu


Abstract
We present a modular, end-to-end dialogue system for a situated agent to address a multimodal, natural language dialogue task in which the agent learns complex representations of block structure classes through assertions, demonstrations, and questioning. The concept to learn is provided to the user through a set of positive and negative visual examples, from which the user determines the underlying constraints to be provided to the system in natural language. The system in turn asks questions about demonstrated examples and simulates new examples to check its knowledge and verify the user’s description is complete. We find that this task is non-trivial for users and generates natural language that is varied yet understood by our deep language understanding architecture.
Anthology ID:
W18-5010
Volume:
Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Kazunori Komatani, Diane Litman, Kai Yu, Alex Papangelis, Lawrence Cavedon, Mikio Nakano
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
89–98
Language:
URL:
https://aclanthology.org/W18-5010
DOI:
10.18653/v1/W18-5010
Bibkey:
Cite (ACL):
Ian Perera, James Allen, Choh Man Teng, and Lucian Galescu. 2018. A Situated Dialogue System for Learning Structural Concepts in Blocks World. In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, pages 89–98, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
A Situated Dialogue System for Learning Structural Concepts in Blocks World (Perera et al., SIGDIAL 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-5010.pdf