Exploring Pre-Trained Transformers and Bilingual Transfer Learning for Arabic Coreference Resolution

Bonan Min


Abstract
In this paper, we develop bilingual transfer learning approaches to improve Arabic coreference resolution by leveraging additional English annotation via bilingual or multilingual pre-trained transformers. We show that bilingual transfer learning improves the strong transformer-based neural coreference models by 2-4 F1. We also systemically investigate the effectiveness of several pre-trained transformer models that differ in training corpora, languages covered, and model capacity. Our best model achieves a new state-of-the-art performance of 64.55 F1 on the Arabic OntoNotes dataset. Our code is publicly available at https://github.com/bnmin/arabic_coref.
Anthology ID:
2021.crac-1.10
Volume:
Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Maciej Ogrodniczuk, Sameer Pradhan, Massimo Poesio, Yulia Grishina, Vincent Ng
Venue:
CRAC
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
94–99
Language:
URL:
https://aclanthology.org/2021.crac-1.10
DOI:
10.18653/v1/2021.crac-1.10
Bibkey:
Cite (ACL):
Bonan Min. 2021. Exploring Pre-Trained Transformers and Bilingual Transfer Learning for Arabic Coreference Resolution. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 94–99, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Exploring Pre-Trained Transformers and Bilingual Transfer Learning for Arabic Coreference Resolution (Min, CRAC 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.crac-1.10.pdf
Video:
 https://aclanthology.org/2021.crac-1.10.mp4
Code
 bnmin/arabic_coref