Interpretable Word Embeddings via Informative Priors

Miriam Hurtado Bodell, Martin Arvidsson, Måns Magnusson


Abstract
Word embeddings have demonstrated strong performance on NLP tasks. However, lack of interpretability and the unsupervised nature of word embeddings have limited their use within computational social science and digital humanities. We propose the use of informative priors to create interpretable and domain-informed dimensions for probabilistic word embeddings. Experimental results show that sensible priors can capture latent semantic concepts better than or on-par with the current state of the art, while retaining the simplicity and generalizability of using priors.
Anthology ID:
D19-1661
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
6323–6329
Language:
URL:
https://aclanthology.org/D19-1661
DOI:
10.18653/v1/D19-1661
Bibkey:
Cite (ACL):
Miriam Hurtado Bodell, Martin Arvidsson, and Måns Magnusson. 2019. Interpretable Word Embeddings via Informative Priors. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6323–6329, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Interpretable Word Embeddings via Informative Priors (Hurtado Bodell et al., EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-1661.pdf
Attachment:
 D19-1661.Attachment.zip