Variational Inference and Deep Generative Models

Wilker Aziz, Philip Schulz


Abstract
NLP has seen a surge in neural network models in recent years. These models provide state-of-the-art performance on many supervised tasks. Unsupervised and semi-supervised learning has only been addressed scarcely, however. Deep generative models (DGMs) make it possible to integrate neural networks with probabilistic graphical models. Using DGMs one can easily design latent variable models that account for missing observations and thereby enable unsupervised and semi-supervised learning with neural networks. The method of choice for training these models is variational inference. This tutorial offers a general introduction to variational inference followed by a thorough and example-driven discussion of how to use variational methods for training DGMs. It provides both the mathematical background necessary for deriving the learning algorithms as well as practical implementation guidelines. Importantly, the tutorial will cover models with continuous and discrete variables. We provide practical coding exercises implemented in IPython notebooks as well as short notes on the more intricate mathematical details that the audience can use as a reference after the tutorial. We expect that with these additional materials the tutorial will have a long-lasting impact on the community.
Anthology ID:
P18-5003
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Yoav Artzi, Jacob Eisenstein
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8–9
Language:
URL:
https://aclanthology.org/P18-5003
DOI:
10.18653/v1/P18-5003
Bibkey:
Cite (ACL):
Wilker Aziz and Philip Schulz. 2018. Variational Inference and Deep Generative Models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pages 8–9, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Variational Inference and Deep Generative Models (Aziz & Schulz, ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/P18-5003.pdf