PanGEA: The Panoramic Graph Environment Annotation Toolkit

Alexander Ku, Peter Anderson, Jordi Pont Tuset, Jason Baldridge


Abstract
PanGEA, the Panoramic Graph Environment Annotation toolkit, is a lightweight toolkit for collecting speech and text annotations in photo-realistic 3D environments. PanGEA immerses annotators in a web-based simulation and allows them to move around easily as they speak and/or listen. It includes database and cloud storage integration, plus utilities for automatically aligning recorded speech with manual transcriptions and the virtual pose of the annotators. Out of the box, PanGEA supports two tasks – collecting navigation instructions and navigation instruction following – and it could be easily adapted for annotating walking tours, finding and labeling landmarks or objects, and similar tasks. We share best practices learned from using PanGEA in a 20,000 hour annotation effort to collect the Room-Across-Room dataset. We hope that our open-source annotation toolkit and insights will both expedite future data collection efforts and spur innovation on the kinds of grounded language tasks such environments can support.
Anthology ID:
2021.alvr-1.5
Volume:
Proceedings of the Second Workshop on Advances in Language and Vision Research
Month:
June
Year:
2021
Address:
Online
Editors:
Xin, Ronghang Hu, Drew Hudson, Tsu-Jui Fu, Marcus Rohrbach, Daniel Fried
Venue:
ALVR
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
29–33
Language:
URL:
https://aclanthology.org/2021.alvr-1.5
DOI:
10.18653/v1/2021.alvr-1.5
Bibkey:
Cite (ACL):
Alexander Ku, Peter Anderson, Jordi Pont Tuset, and Jason Baldridge. 2021. PanGEA: The Panoramic Graph Environment Annotation Toolkit. In Proceedings of the Second Workshop on Advances in Language and Vision Research, pages 29–33, Online. Association for Computational Linguistics.
Cite (Informal):
PanGEA: The Panoramic Graph Environment Annotation Toolkit (Ku et al., ALVR 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.alvr-1.5.pdf
Data
Matterport3DRxR