Situated Dialogue Learning through Procedural Environment Generation

Prithviraj Ammanabrolu, Renee Jia, Mark Riedl


Abstract
We teach goal-driven agents to interactively act and speak in situated environments by training on generated curriculums. Our agents operate in LIGHT (Urbanek et al. 2019)—a large-scale crowd-sourced fantasy text adventure game wherein an agent perceives and interacts with the world through textual natural language. Goals in this environment take the form of character-based quests, consisting of personas and motivations. We augment LIGHT by learning to procedurally generate additional novel textual worlds and quests to create a curriculum of steadily increasing difficulty for training agents to achieve such goals. In particular, we measure curriculum difficulty in terms of the rarity of the quest in the original training distribution—an easier environment is one that is more likely to have been found in the unaugmented dataset. An ablation study shows that this method of learning from the tail of a distribution results in significantly higher generalization abilities as measured by zero-shot performance on never-before-seen quests.
Anthology ID:
2022.acl-long.557
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8099–8116
Language:
URL:
https://aclanthology.org/2022.acl-long.557
DOI:
10.18653/v1/2022.acl-long.557
Bibkey:
Cite (ACL):
Prithviraj Ammanabrolu, Renee Jia, and Mark Riedl. 2022. Situated Dialogue Learning through Procedural Environment Generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8099–8116, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Situated Dialogue Learning through Procedural Environment Generation (Ammanabrolu et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.557.pdf
Video:
 https://aclanthology.org/2022.acl-long.557.mp4