Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning

Shuoran Jiang, Qingcai Chen, Xin Liu, Baotian Hu, Lisai Zhang


Abstract
Graph convolutional network (GCN) has become popular in various natural language processing (NLP) tasks with its superiority in long-term and non-consecutive word interactions. However, existing single-hop graph reasoning in GCN may miss some important non-consecutive dependencies. In this study, we define the spectral graph convolutional network with the high-order dynamic Chebyshev approximation (HDGCN), which augments the multi-hop graph reasoning by fusing messages aggregated from direct and long-term dependencies into one convolutional layer. To alleviate the over-smoothing in high-order Chebyshev approximation, a multi-vote-based cross-attention (MVCAttn) with linear computation complexity is also proposed. The empirical results on four transductive and inductive NLP tasks and the ablation study verify the efficacy of the proposed model.
Anthology ID:
2021.acl-long.513
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Editors:
Chengqing Zong, Fei Xia, Wenjie Li, Roberto Navigli
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6563–6573
Language:
URL:
https://aclanthology.org/2021.acl-long.513
DOI:
10.18653/v1/2021.acl-long.513
Bibkey:
Cite (ACL):
Shuoran Jiang, Qingcai Chen, Xin Liu, Baotian Hu, and Lisai Zhang. 2021. Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6563–6573, Online. Association for Computational Linguistics.
Cite (Informal):
Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning (Jiang et al., ACL-IJCNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-long.513.pdf
Video:
 https://aclanthology.org/2021.acl-long.513.mp4
Code
 MathIsAll/HDGCN-pytorch
Data
SNLI