¿Es un plátano? Exploring the Application of a Physically Grounded Language Acquisition System to Spanish

Caroline Kery, Francis Ferraro, Cynthia Matuszek


Abstract
In this paper we describe a multilingual grounded language learning system adapted from an English-only system. This system learns the meaning of words used in crowd-sourced descriptions by grounding them in the physical representations of the objects they are describing. Our work presents a framework to compare the performance of the system when applied to a new language and to identify modifications necessary to attain equal performance, with the goal of enhancing the ability of robots to learn language from a more diverse range of people. We then demonstrate this system with Spanish, through first analyzing the performance of translated Spanish, and then extending this analysis to a new corpus of crowd-sourced Spanish language data. We find that with small modifications, the system is able to learn color, object, and shape words with comparable performance between languages.
Anthology ID:
W19-1602
Volume:
Proceedings of the Combined Workshop on Spatial Language Understanding (SpLU) and Grounded Communication for Robotics (RoboNLP)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Archna Bhatia, Yonatan Bisk, Parisa Kordjamshidi, Jesse Thomason
Venue:
RoboNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7–17
Language:
URL:
https://aclanthology.org/W19-1602
DOI:
10.18653/v1/W19-1602
Bibkey:
Cite (ACL):
Caroline Kery, Francis Ferraro, and Cynthia Matuszek. 2019. ¿Es un plátano? Exploring the Application of a Physically Grounded Language Acquisition System to Spanish. In Proceedings of the Combined Workshop on Spatial Language Understanding (SpLU) and Grounded Communication for Robotics (RoboNLP), pages 7–17, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
¿Es un plátano? Exploring the Application of a Physically Grounded Language Acquisition System to Spanish (Kery et al., RoboNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-1602.pdf