Multi-level Community-awareness Graph Neural Networks for Neural Machine Translation

Binh Nguyen, Long Nguyen, Dien Dinh


Abstract
Neural Machine Translation (NMT) aims to translate the source- to the target-language while preserving the original meaning. Linguistic information such as morphology, syntactic, and semantics shall be grasped in token embeddings to produce a high-quality translation. Recent works have leveraged the powerful Graph Neural Networks (GNNs) to encode such language knowledge into token embeddings. Specifically, they use a trained parser to construct semantic graphs given sentences and then apply GNNs. However, most semantic graphs are tree-shaped and too sparse for GNNs which cause the over-smoothing problem. To alleviate this problem, we propose a novel Multi-level Community-awareness Graph Neural Network (MC-GNN) layer to jointly model local and global relationships between words and their linguistic roles in multiple communities. Intuitively, the MC-GNN layer substitutes a self-attention layer at the encoder side of a transformer-based machine translation model. Extensive experiments on four language-pair datasets with common evaluation metrics show the remarkable improvements of our method while reducing the time complexity in very long sentences.
Anthology ID:
2022.coling-1.444
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
5021–5028
Language:
URL:
https://aclanthology.org/2022.coling-1.444
DOI:
Bibkey:
Cite (ACL):
Binh Nguyen, Long Nguyen, and Dien Dinh. 2022. Multi-level Community-awareness Graph Neural Networks for Neural Machine Translation. In Proceedings of the 29th International Conference on Computational Linguistics, pages 5021–5028, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
Multi-level Community-awareness Graph Neural Networks for Neural Machine Translation (Nguyen et al., COLING 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.coling-1.444.pdf
Code
 nqbinh17/mc-gnn