%0 Journal Article %T Relational Memory-Augmented Language Models %A Liu, Qi %A Yogatama, Dani %A Blunsom, Phil %J Transactions of the Association for Computational Linguistics %D 2022 %V 10 %I MIT Press %C Cambridge, MA %F liu-etal-2022-relational %X We present a memory-augmented approach to condition an autoregressive language model on a knowledge graph. We represent the graph as a collection of relation triples and retrieve relevant relations for a given context to improve text generation. Experiments on WikiText-103, WMT19, and enwik8 English datasets demonstrate that our approach produces a better language model in terms of perplexity and bits per character. We also show that relational memory improves coherence, is complementary to token-based memory, and enables causal interventions. Our model provides a simple yet effective way to combine an autoregressive language model and a knowledge graph for more coherent and logical generation. %R 10.1162/tacl_a_00476 %U https://aclanthology.org/2022.tacl-1.32 %U https://doi.org/10.1162/tacl_a_00476 %P 555-572