Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations

Christian Hadiwinoto, Hwee Tou Ng, Wee Chung Gan


Abstract
Contextualized word representations are able to give different representations for the same word in different contexts, and they have been shown to be effective in downstream natural language processing tasks, such as question answering, named entity recognition, and sentiment analysis. However, evaluation on word sense disambiguation (WSD) in prior work shows that using contextualized word representations does not outperform the state-of-the-art approach that makes use of non-contextualized word embeddings. In this paper, we explore different strategies of integrating pre-trained contextualized word representations and our best strategy achieves accuracies exceeding the best prior published accuracies by significant margins on multiple benchmark WSD datasets.
Anthology ID:
D19-1533
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
5297–5306
Language:
URL:
https://aclanthology.org/D19-1533
DOI:
10.18653/v1/D19-1533
Bibkey:
Cite (ACL):
Christian Hadiwinoto, Hwee Tou Ng, and Wee Chung Gan. 2019. Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5297–5306, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (Hadiwinoto et al., EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-1533.pdf
Code
 nusnlp/contextemb-wsd
Data
OntoNotes 5.0Word Sense Disambiguation: a Unified Evaluation Framework and Empirical Comparison