Probing sentence embeddings for structure-dependent tense

Geoff Bacon, Terry Regier


Abstract
Learning universal sentence representations which accurately model sentential semantic content is a current goal of natural language processing research. A prominent and successful approach is to train recurrent neural networks (RNNs) to encode sentences into fixed length vectors. Many core linguistic phenomena that one would like to model in universal sentence representations depend on syntactic structure. Despite the fact that RNNs do not have explicit syntactic structural representations, there is some evidence that RNNs can approximate such structure-dependent phenomena under certain conditions, in addition to their widespread success in practical tasks. In this work, we assess RNNs’ ability to learn the structure-dependent phenomenon of main clause tense.
Anthology ID:
W18-5440
Volume:
Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
Month:
November
Year:
2018
Address:
Brussels, Belgium
Editors:
Tal Linzen, Grzegorz Chrupała, Afra Alishahi
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
334–336
Language:
URL:
https://aclanthology.org/W18-5440
DOI:
10.18653/v1/W18-5440
Bibkey:
Cite (ACL):
Geoff Bacon and Terry Regier. 2018. Probing sentence embeddings for structure-dependent tense. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 334–336, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Probing sentence embeddings for structure-dependent tense (Bacon & Regier, EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-5440.pdf
Data
Universal Dependencies