Transfer in Deep Reinforcement Learning Using Knowledge Graphs

Prithviraj Ammanabrolu, Mark Riedl


Abstract
Text adventure games, in which players must make sense of the world through text descriptions and declare actions through text descriptions, provide a stepping stone toward grounding action in language. Prior work has demonstrated that using a knowledge graph as a state representation and question-answering to pre-train a deep Q-network facilitates faster control policy learning. In this paper, we explore the use of knowledge graphs as a representation for domain knowledge transfer for training text-adventure playing reinforcement learning agents. Our methods are tested across multiple computer generated and human authored games, varying in domain and complexity, and demonstrate that our transfer learning methods let us learn a higher-quality control policy faster.
Anthology ID:
D19-5301
Volume:
Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13)
Month:
November
Year:
2019
Address:
Hong Kong
Editors:
Dmitry Ustalov, Swapna Somasundaran, Peter Jansen, Goran Glavaš, Martin Riedl, Mihai Surdeanu, Michalis Vazirgiannis
Venue:
TextGraphs
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–10
Language:
URL:
https://aclanthology.org/D19-5301
DOI:
10.18653/v1/D19-5301
Bibkey:
Cite (ACL):
Prithviraj Ammanabrolu and Mark Riedl. 2019. Transfer in Deep Reinforcement Learning Using Knowledge Graphs. In Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13), pages 1–10, Hong Kong. Association for Computational Linguistics.
Cite (Informal):
Transfer in Deep Reinforcement Learning Using Knowledge Graphs (Ammanabrolu & Riedl, TextGraphs 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-5301.pdf