The Impact of Edge Displacement Vaserstein Distance on UD Parsing Performance

Mark Anderson, Carlos Gómez-Rodríguez


Abstract
We contribute to the discussion on parsing performance in NLP by introducing a measurement that evaluates the differences between the distributions of edge displacement (the directed distance of edges) seen in training and test data. We hypothesize that this measurement will be related to differences observed in parsing performance across treebanks. We motivate this by building upon previous work and then attempt to falsify this hypothesis by using a number of statistical methods. We establish that there is a statistical correlation between this measurement and parsing performance even when controlling for potential covariants. We then use this to establish a sampling technique that gives us an adversarial and complementary split. This gives an idea of the lower and upper bounds of parsing systems for a given treebank in lieu of freshly sampled data. In a broader sense, the methodology presented here can act as a reference for future correlation-based exploratory work in NLP.
Anthology ID:
2022.cl-3.2
Volume:
Computational Linguistics, Volume 48, Issue 3 - September 2022
Month:
September
Year:
2022
Address:
Cambridge, MA
Venue:
CL
SIG:
Publisher:
MIT Press
Note:
Pages:
517–554
Language:
URL:
https://aclanthology.org/2022.cl-3.2
DOI:
10.1162/coli_a_00440
Bibkey:
Cite (ACL):
Mark Anderson and Carlos Gómez-Rodríguez. 2022. The Impact of Edge Displacement Vaserstein Distance on UD Parsing Performance. Computational Linguistics, 48(3):517–554.
Cite (Informal):
The Impact of Edge Displacement Vaserstein Distance on UD Parsing Performance (Anderson & Gómez-Rodríguez, CL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.cl-3.2.pdf
Video:
 https://aclanthology.org/2022.cl-3.2.mp4
Code
 markda/morphological-complexity