Towards Replicability in Parsing

Daniel Dakota, Sandra Kübler


Abstract
We investigate parsing replicability across 7 languages (and 8 treebanks), showing that choices concerning the use of grammatical functions in parsing or evaluation, the influence of the rare word threshold, as well as choices in test sentences and evaluation script options have considerable and often unexpected effects on parsing accuracies. All of those choices need to be carefully documented if we want to ensure replicability.
Anthology ID:
R17-1026
Volume:
Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017
Month:
September
Year:
2017
Address:
Varna, Bulgaria
Editors:
Ruslan Mitkov, Galia Angelova
Venue:
RANLP
SIG:
Publisher:
INCOMA Ltd.
Note:
Pages:
185–194
Language:
URL:
https://doi.org/10.26615/978-954-452-049-6_026
DOI:
10.26615/978-954-452-049-6_026
Bibkey:
Cite (ACL):
Daniel Dakota and Sandra Kübler. 2017. Towards Replicability in Parsing. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pages 185–194, Varna, Bulgaria. INCOMA Ltd..
Cite (Informal):
Towards Replicability in Parsing (Dakota & Kübler, RANLP 2017)
Copy Citation:
PDF:
https://doi.org/10.26615/978-954-452-049-6_026