KW-ATTN: Knowledge Infused Attention for Accurate and Interpretable Text Classification

Hyeju Jang, Seojin Bang, Wen Xiao, Giuseppe Carenini, Raymond Ng, Young ji Lee


Abstract
Text classification has wide-ranging applications in various domains. While neural network approaches have drastically advanced performance in text classification, they tend to be powered by a large amount of training data, and interpretability is often an issue. As a step towards better accuracy and interpretability especially on small data, in this paper we present a new knowledge-infused attention mechanism, called KW-ATTN (KnoWledge-infused ATTentioN) to incorporate high-level concepts from external knowledge bases into Neural Network models. We show that KW-ATTN outperforms baseline models using only words as well as other approaches using concepts by classification accuracy, which indicates that high-level concepts help model prediction. Furthermore, crowdsourced human evaluation suggests that additional concept information helps interpretability of the model.
Anthology ID:
2021.deelio-1.10
Volume:
Proceedings of Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures
Month:
June
Year:
2021
Address:
Online
Venues:
DeeLIO | NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
96–107
Language:
URL:
https://aclanthology.org/2021.deelio-1.10
DOI:
10.18653/v1/2021.deelio-1.10
Bibkey:
Cite (ACL):
Hyeju Jang, Seojin Bang, Wen Xiao, Giuseppe Carenini, Raymond Ng, and Young ji Lee. 2021. KW-ATTN: Knowledge Infused Attention for Accurate and Interpretable Text Classification. In Proceedings of Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 96–107, Online. Association for Computational Linguistics.
Cite (Informal):
KW-ATTN: Knowledge Infused Attention for Accurate and Interpretable Text Classification (Jang et al., DeeLIO 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.deelio-1.10.pdf