MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks

Cristian-Paul Bara, Sky CH-Wang, Joyce Chai


Abstract
An ideal integration of autonomous agents in a human world implies that they are able to collaborate on human terms. In particular, theory of mind plays an important role in maintaining common ground during human collaboration and communication. To enable theory of mind modeling in situated interactions, we introduce a fine-grained dataset of collaborative tasks performed by pairs of human subjects in the 3D virtual blocks world of Minecraft. It provides information that captures partners’ beliefs of the world and of each other as an interaction unfolds, bringing abundant opportunities to study human collaborative behaviors in situated language communication. As a first step towards our goal of developing embodied AI agents able to infer belief states of collaborative partners in situ, we build and present results on computational models for several theory of mind tasks.
Anthology ID:
2021.emnlp-main.85
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1112–1125
Language:
URL:
https://aclanthology.org/2021.emnlp-main.85
DOI:
10.18653/v1/2021.emnlp-main.85
Bibkey:
Cite (ACL):
Cristian-Paul Bara, Sky CH-Wang, and Joyce Chai. 2021. MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1112–1125, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks (Bara et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.85.pdf
Video:
 https://aclanthology.org/2021.emnlp-main.85.mp4
Data
MindCraft