Gaussian Mixture Latent Vector Grammars

Yanpeng Zhao, Liwen Zhang, Kewei Tu


Abstract
We introduce Latent Vector Grammars (LVeGs), a new framework that extends latent variable grammars such that each nonterminal symbol is associated with a continuous vector space representing the set of (infinitely many) subtypes of the nonterminal. We show that previous models such as latent variable grammars and compositional vector grammars can be interpreted as special cases of LVeGs. We then present Gaussian Mixture LVeGs (GM-LVeGs), a new special case of LVeGs that uses Gaussian mixtures to formulate the weights of production rules over subtypes of nonterminals. A major advantage of using Gaussian mixtures is that the partition function and the expectations of subtype rules can be computed using an extension of the inside-outside algorithm, which enables efficient inference and learning. We apply GM-LVeGs to part-of-speech tagging and constituency parsing and show that GM-LVeGs can achieve competitive accuracies.
Anthology ID:
P18-1109
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1181–1189
Language:
URL:
https://aclanthology.org/P18-1109
DOI:
10.18653/v1/P18-1109
Bibkey:
Cite (ACL):
Yanpeng Zhao, Liwen Zhang, and Kewei Tu. 2018. Gaussian Mixture Latent Vector Grammars. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1181–1189, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Gaussian Mixture Latent Vector Grammars (Zhao et al., ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/P18-1109.pdf
Note:
 P18-1109.Notes.zip
Video:
 https://aclanthology.org/P18-1109.mp4
Code
 zhaoyanpeng/lveg
Data
Universal Dependencies