Translating Neuralese

Jacob Andreas, Anca Dragan, Dan Klein


Abstract
Several approaches have recently been proposed for learning decentralized deep multiagent policies that coordinate via a differentiable communication channel. While these policies are effective for many tasks, interpretation of their induced communication strategies has remained a challenge. Here we propose to interpret agents’ messages by translating them. Unlike in typical machine translation problems, we have no parallel data to learn from. Instead we develop a translation model based on the insight that agent messages and natural language strings mean the same thing if they induce the same belief about the world in a listener. We present theoretical guarantees and empirical evidence that our approach preserves both the semantics and pragmatics of messages by ensuring that players communicating through a translation layer do not suffer a substantial loss in reward relative to players with a common language.
Anthology ID:
P17-1022
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Editors:
Regina Barzilay, Min-Yen Kan
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
232–242
Language:
URL:
https://aclanthology.org/P17-1022
DOI:
10.18653/v1/P17-1022
Bibkey:
Cite (ACL):
Jacob Andreas, Anca Dragan, and Dan Klein. 2017. Translating Neuralese. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 232–242, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Translating Neuralese (Andreas et al., ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-1022.pdf
Note:
 P17-1022.Notes.pdf
Video:
 https://aclanthology.org/P17-1022.mp4
Code
 jacobandreas/neuralese