Learning beyond Datasets: Knowledge Graph Augmented Neural Networks for Natural Language Processing

Annervaz K M, Somnath Basu Roy Chowdhury, Ambedkar Dukkipati


Abstract
Machine Learning has been the quintessential solution for many AI problems, but learning models are heavily dependent on specific training data. Some learning models can be incorporated with prior knowledge using a Bayesian setup, but these learning models do not have the ability to access any organized world knowledge on demand. In this work, we propose to enhance learning models with world knowledge in the form of Knowledge Graph (KG) fact triples for Natural Language Processing (NLP) tasks. Our aim is to develop a deep learning model that can extract relevant prior support facts from knowledge graphs depending on the task using attention mechanism. We introduce a convolution-based model for learning representations of knowledge graph entity and relation clusters in order to reduce the attention space. We show that the proposed method is highly scalable to the amount of prior information that has to be processed and can be applied to any generic NLP task. Using this method we show significant improvement in performance for text classification with 20Newsgroups (News20) & DBPedia datasets, and natural language inference with Stanford Natural Language Inference (SNLI) dataset. We also demonstrate that a deep learning model can be trained with substantially less amount of labeled training data, when it has access to organized world knowledge in the form of a knowledge base.
Anthology ID:
N18-1029
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Editors:
Marilyn Walker, Heng Ji, Amanda Stent
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
313–322
Language:
URL:
https://aclanthology.org/N18-1029
DOI:
10.18653/v1/N18-1029
Bibkey:
Cite (ACL):
Annervaz K M, Somnath Basu Roy Chowdhury, and Ambedkar Dukkipati. 2018. Learning beyond Datasets: Knowledge Graph Augmented Neural Networks for Natural Language Processing. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 313–322, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Learning beyond Datasets: Knowledge Graph Augmented Neural Networks for Natural Language Processing (K M et al., NAACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/N18-1029.pdf
Video:
 https://aclanthology.org/N18-1029.mp4
Data
FB15kSNLIWN18