Structured Generative Models of Continuous Features for Word Sense Induction

Alexandros Komninos, Suresh Manandhar


Abstract
We propose a structured generative latent variable model that integrates information from multiple contextual representations for Word Sense Induction. Our approach jointly models global lexical, local lexical and dependency syntactic context. Each context type is associated with a latent variable and the three types of variables share a hierarchical structure. We use skip-gram based word and dependency context embeddings to construct all three types of representations, reducing the total number of parameters to be estimated and enabling better generalization. We describe an EM algorithm to efficiently estimate model parameters and use the Integrated Complete Likelihood criterion to automatically estimate the number of senses. Our model achieves state-of-the-art results on the SemEval-2010 and SemEval-2013 Word Sense Induction datasets.
Anthology ID:
C16-1337
Volume:
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Yuji Matsumoto, Rashmi Prasad
Venue:
COLING
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
3577–3587
Language:
URL:
https://aclanthology.org/C16-1337
DOI:
Bibkey:
Cite (ACL):
Alexandros Komninos and Suresh Manandhar. 2016. Structured Generative Models of Continuous Features for Word Sense Induction. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3577–3587, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
Structured Generative Models of Continuous Features for Word Sense Induction (Komninos & Manandhar, COLING 2016)
Copy Citation:
PDF:
https://aclanthology.org/C16-1337.pdf