Parameter sharing between dependency parsers for related languages

Miryam de Lhoneux, Johannes Bjerva, Isabelle Augenstein, Anders Søgaard


Abstract
Previous work has suggested that parameter sharing between transition-based neural dependency parsers for related languages can lead to better performance, but there is no consensus on what parameters to share. We present an evaluation of 27 different parameter sharing strategies across 10 languages, representing five pairs of related languages, each pair from a different language family. We find that sharing transition classifier parameters always helps, whereas the usefulness of sharing word and/or character LSTM parameters varies. Based on this result, we propose an architecture where the transition classifier is shared, and the sharing of word and character parameters is controlled by a parameter that can be tuned on validation data. This model is linguistically motivated and obtains significant improvements over a monolingually trained baseline. We also find that sharing transition classifier parameters helps when training a parser on unrelated language pairs, but we find that, in the case of unrelated languages, sharing too many parameters does not help.
Anthology ID:
D18-1543
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Editors:
Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
4992–4997
Language:
URL:
https://aclanthology.org/D18-1543
DOI:
10.18653/v1/D18-1543
Bibkey:
Cite (ACL):
Miryam de Lhoneux, Johannes Bjerva, Isabelle Augenstein, and Anders Søgaard. 2018. Parameter sharing between dependency parsers for related languages. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4992–4997, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Parameter sharing between dependency parsers for related languages (de Lhoneux et al., EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/D18-1543.pdf
Attachment:
 D18-1543.Attachment.pdf
Poster:
 D18-1543.Poster.pdf
Code
 coastalcph/uuparser
Data
Universal Dependencies