On Learning Interpreted Languages with Recurrent Models

Denis Paperno


Abstract
Can recurrent neural nets, inspired by human sequential data processing, learn to understand language? We construct simplified data sets reflecting core properties of natural language as modeled in formal syntax and semantics: recursive syntactic structure and compositionality. We find LSTM and GRU networks to generalize to compositional interpretation well, but only in the most favorable learning settings, with a well-paced curriculum, extensive training data, and left-to-right (but not right-to-left) composition.
Anthology ID:
2022.cl-2.7
Volume:
Computational Linguistics, Volume 48, Issue 2 - June 2022
Month:
June
Year:
2022
Address:
Cambridge, MA
Venue:
CL
SIG:
Publisher:
MIT Press
Note:
Pages:
471–482
Language:
URL:
https://aclanthology.org/2022.cl-2.7
DOI:
10.1162/coli_a_00431
Bibkey:
Cite (ACL):
Denis Paperno. 2022. On Learning Interpreted Languages with Recurrent Models. Computational Linguistics, 48(2):471–482.
Cite (Informal):
On Learning Interpreted Languages with Recurrent Models (Paperno, CL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.cl-2.7.pdf