Test Harder than You Train: Probing with Extrapolation Splits

Jenny Kunz, Marco Kuhlmann


Abstract
Previous work on probing word representations for linguistic knowledge has focused on interpolation tasks. In this paper, we instead analyse probes in an extrapolation setting, where the inputs at test time are deliberately chosen to be ‘harder’ than the training examples. We argue that such an analysis can shed further light on the open question whether probes actually decode linguistic knowledge, or merely learn the diagnostic task from shallow features. To quantify the hardness of an example, we consider scoring functions based on linguistic, statistical, and learning-related criteria, all of which are applicable to a broad range of NLP tasks. We discuss the relative merits of these criteria in the context of two syntactic probing tasks, part-of-speech tagging and syntactic dependency labelling. From our theoretical and experimental analysis, we conclude that distance-based and hard statistical criteria show the clearest differences between interpolation and extrapolation settings, while at the same time being transparent, intuitive, and easy to control.
Anthology ID:
2021.blackboxnlp-1.2
Volume:
Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Jasmijn Bastings, Yonatan Belinkov, Emmanuel Dupoux, Mario Giulianelli, Dieuwke Hupkes, Yuval Pinter, Hassan Sajjad
Venue:
BlackboxNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
15–25
Language:
URL:
https://aclanthology.org/2021.blackboxnlp-1.2
DOI:
10.18653/v1/2021.blackboxnlp-1.2
Bibkey:
Cite (ACL):
Jenny Kunz and Marco Kuhlmann. 2021. Test Harder than You Train: Probing with Extrapolation Splits. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 15–25, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Test Harder than You Train: Probing with Extrapolation Splits (Kunz & Kuhlmann, BlackboxNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.blackboxnlp-1.2.pdf
Software:
 2021.blackboxnlp-1.2.Software.zip
Video:
 https://aclanthology.org/2021.blackboxnlp-1.2.mp4
Code
 jekunz/extrapolation