Cross-lingual tagger evaluation without test data

Željko Agić, Barbara Plank, Anders Søgaard


Abstract
We address the challenge of cross-lingual POS tagger evaluation in absence of manually annotated test data. We put forth and evaluate two dictionary-based metrics. On the tasks of accuracy prediction and system ranking, we reveal that these metrics are reliable enough to approximate test set-based evaluation, and at the same time lean enough to support assessment for truly low-resource languages.
Anthology ID:
E17-2040
Volume:
Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers
Month:
April
Year:
2017
Address:
Valencia, Spain
Editors:
Mirella Lapata, Phil Blunsom, Alexander Koller
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
248–253
Language:
URL:
https://aclanthology.org/E17-2040
DOI:
Bibkey:
Cite (ACL):
Željko Agić, Barbara Plank, and Anders Søgaard. 2017. Cross-lingual tagger evaluation without test data. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 248–253, Valencia, Spain. Association for Computational Linguistics.
Cite (Informal):
Cross-lingual tagger evaluation without test data (Agić et al., EACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/E17-2040.pdf