Crosslingual Transfer Learning for Relation and Event Extraction via Word Category and Class Alignments

Minh Van Nguyen, Tuan Ngo Nguyen, Bonan Min, Thien Huu Nguyen


Abstract
Previous work on crosslingual Relation and Event Extraction (REE) suffers from the monolingual bias issue due to the training of models on only the source language data. An approach to overcome this issue is to use unlabeled data in the target language to aid the alignment of crosslingual representations, i.e., via fooling a language discriminator. However, as this approach does not condition on class information, a target language example of a class could be incorrectly aligned to a source language example of a different class. To address this issue, we propose a novel crosslingual alignment method that leverages class information of REE tasks for representation learning. In particular, we propose to learn two versions of representation vectors for each class in an REE task based on either source or target language examples. Representation vectors for corresponding classes will then be aligned to achieve class-aware alignment for crosslingual representations. In addition, we propose to further align representation vectors for language-universal word categories (i.e., parts of speech and dependency relations). As such, a novel filtering mechanism is presented to facilitate the learning of word category representations from contextualized representations on input texts based on adversarial learning. We conduct extensive crosslingual experiments with English, Chinese, and Arabic over REE tasks. The results demonstrate the benefits of the proposed method that significantly advances the state-of-the-art performance in these settings.
Anthology ID:
2021.emnlp-main.440
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5414–5426
Language:
URL:
https://aclanthology.org/2021.emnlp-main.440
DOI:
10.18653/v1/2021.emnlp-main.440
Bibkey:
Cite (ACL):
Minh Van Nguyen, Tuan Ngo Nguyen, Bonan Min, and Thien Huu Nguyen. 2021. Crosslingual Transfer Learning for Relation and Event Extraction via Word Category and Class Alignments. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5414–5426, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Crosslingual Transfer Learning for Relation and Event Extraction via Word Category and Class Alignments (Nguyen et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.440.pdf
Video:
 https://aclanthology.org/2021.emnlp-main.440.mp4