Learning to Read Maps: Understanding Natural Language Instructions from Unseen Maps

Miltiadis Marios Katsakioris, Ioannis Konstas, Pierre Yves Mignotte, Helen Hastie


Abstract
Robust situated dialog requires the ability to process instructions based on spatial information, which may or may not be available. We propose a model, based on LXMERT, that can extract spatial information from text instructions and attend to landmarks on OpenStreetMap (OSM) referred to in a natural language instruction. Whilst, OSM is a valuable resource, as with any open-sourced data, there is noise and variation in the names referred to on the map, as well as, variation in natural language instructions, hence the need for data-driven methods over rule-based systems. This paper demonstrates that the gold GPS location can be accurately predicted from the natural language instruction and metadata with 72% accuracy for previously seen maps and 64% for unseen maps.
Anthology ID:
2021.splurobonlp-1.2
Volume:
Proceedings of Second International Combined Workshop on Spatial Language Understanding and Grounded Communication for Robotics
Month:
August
Year:
2021
Address:
Online
Editors:
Malihe Alikhani, Valts Blukis, Parisa Kordjamshidi, Aishwarya Padmakumar, Hao Tan
Venue:
splurobonlp
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11–21
Language:
URL:
https://aclanthology.org/2021.splurobonlp-1.2
DOI:
10.18653/v1/2021.splurobonlp-1.2
Bibkey:
Cite (ACL):
Miltiadis Marios Katsakioris, Ioannis Konstas, Pierre Yves Mignotte, and Helen Hastie. 2021. Learning to Read Maps: Understanding Natural Language Instructions from Unseen Maps. In Proceedings of Second International Combined Workshop on Spatial Language Understanding and Grounded Communication for Robotics, pages 11–21, Online. Association for Computational Linguistics.
Cite (Informal):
Learning to Read Maps: Understanding Natural Language Instructions from Unseen Maps (Katsakioris et al., splurobonlp 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.splurobonlp-1.2.pdf
Code
 marioskatsak/mapert