Bayesian Learning of Latent Representations of Language Structures

Yugo Murawaki


Abstract
We borrow the concept of representation learning from deep learning research, and we argue that the quest for Greenbergian implicational universals can be reformulated as the learning of good latent representations of languages, or sequences of surface typological features. By projecting languages into latent representations and performing inference in the latent space, we can handle complex dependencies among features in an implicit manner. The most challenging problem in turning the idea into a concrete computational model is the alarmingly large number of missing values in existing typological databases. To address this problem, we keep the number of model parameters relatively small to avoid overfitting, adopt the Bayesian learning framework for its robustness, and exploit phylogenetically and/or spatially related languages as additional clues. Experiments show that the proposed model recovers missing values more accurately than others and that some latent variables exhibit phylogenetic and spatial signals comparable to those of surface features.
Anthology ID:
J19-2001
Volume:
Computational Linguistics, Volume 45, Issue 2 - June 2019
Month:
June
Year:
2019
Address:
Cambridge, MA
Venue:
CL
SIG:
Publisher:
MIT Press
Note:
Pages:
199–228
Language:
URL:
https://aclanthology.org/J19-2001
DOI:
10.1162/coli_a_00346
Bibkey:
Cite (ACL):
Yugo Murawaki. 2019. Bayesian Learning of Latent Representations of Language Structures. Computational Linguistics, 45(2):199–228.
Cite (Informal):
Bayesian Learning of Latent Representations of Language Structures (Murawaki, CL 2019)
Copy Citation:
PDF:
https://aclanthology.org/J19-2001.pdf