Cross-lingual Text Classification with Heterogeneous Graph Neural Network

Ziyun Wang, Xuan Liu, Peiji Yang, Shixing Liu, Zhisheng Wang


Abstract
Cross-lingual text classification aims at training a classifier on the source language and transferring the knowledge to target languages, which is very useful for low-resource languages. Recent multilingual pretrained language models (mPLM) achieve impressive results in cross-lingual classification tasks, but rarely consider factors beyond semantic similarity, causing performance degradation between some language pairs. In this paper we propose a simple yet effective method to incorporate heterogeneous information within and across languages for cross-lingual text classification using graph convolutional networks (GCN). In particular, we construct a heterogeneous graph by treating documents and words as nodes, and linking nodes with different relations, which include part-of-speech roles, semantic similarity, and document translations. Extensive experiments show that our graph-based method significantly outperforms state-of-the-art models on all tasks, and also achieves consistent performance gain over baselines in low-resource settings where external tools like translators are unavailable.
Anthology ID:
2021.acl-short.78
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Month:
August
Year:
2021
Address:
Online
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
612–620
Language:
URL:
https://aclanthology.org/2021.acl-short.78
DOI:
10.18653/v1/2021.acl-short.78
Bibkey:
Cite (ACL):
Ziyun Wang, Xuan Liu, Peiji Yang, Shixing Liu, and Zhisheng Wang. 2021. Cross-lingual Text Classification with Heterogeneous Graph Neural Network. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 612–620, Online. Association for Computational Linguistics.
Cite (Informal):
Cross-lingual Text Classification with Heterogeneous Graph Neural Network (Wang et al., ACL-IJCNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-short.78.pdf
Video:
 https://aclanthology.org/2021.acl-short.78.mp4
Code
 TencentGameMate/gnn_cross_lingual
Data
XGLUE