Unsupervised Online Grounding of Natural Language during Human-Robot Interactions

Oliver Roesler


Abstract
Allowing humans to communicate through natural language with robots requires connections between words and percepts. The process of creating these connections is called symbol grounding and has been studied for nearly three decades. Although many studies have been conducted, not many considered grounding of synonyms and the employed algorithms either work only offline or in a supervised manner. In this paper, a cross-situational learning based grounding framework is proposed that allows grounding of words and phrases through corresponding percepts without human supervision and online, i.e. it does not require any explicit training phase, but instead updates the obtained mappings for every new encountered situation. The proposed framework is evaluated through an interaction experiment between a human tutor and a robot, and compared to an existing unsupervised grounding framework. The results show that the proposed framework is able to ground words through their corresponding percepts online and in an unsupervised manner, while outperforming the baseline framework.
Anthology ID:
2020.challengehml-1.5
Volume:
Second Grand-Challenge and Workshop on Multimodal Language (Challenge-HML)
Month:
July
Year:
2020
Address:
Seattle, USA
Venues:
ACL | Challenge-HML | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
35–45
Language:
URL:
https://aclanthology.org/2020.challengehml-1.5
DOI:
10.18653/v1/2020.challengehml-1.5
Bibkey:
Cite (ACL):
Oliver Roesler. 2020. Unsupervised Online Grounding of Natural Language during Human-Robot Interactions. In Second Grand-Challenge and Workshop on Multimodal Language (Challenge-HML), pages 35–45, Seattle, USA. Association for Computational Linguistics.
Cite (Informal):
Unsupervised Online Grounding of Natural Language during Human-Robot Interactions (Roesler, Challenge-HML 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.challengehml-1.5.pdf
Video:
 http://slideslive.com/38931262