Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences

Boon Peng Yap, Andrew Koh, Eng Siong Chng


Abstract
Domain adaptation or transfer learning using pre-trained language models such as BERT has proven to be an effective approach for many natural language processing tasks. In this work, we propose to formulate word sense disambiguation as a relevance ranking task, and fine-tune BERT on sequence-pair ranking task to select the most probable sense definition given a context sentence and a list of candidate sense definitions. We also introduce a data augmentation technique for WSD using existing example sentences from WordNet. Using the proposed training objective and data augmentation technique, our models are able to achieve state-of-the-art results on the English all-words benchmark datasets.
Anthology ID:
2020.findings-emnlp.4
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2020
Month:
November
Year:
2020
Address:
Online
Editors:
Trevor Cohn, Yulan He, Yang Liu
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
41–46
Language:
URL:
https://aclanthology.org/2020.findings-emnlp.4
DOI:
10.18653/v1/2020.findings-emnlp.4
Bibkey:
Cite (ACL):
Boon Peng Yap, Andrew Koh, and Eng Siong Chng. 2020. Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 41–46, Online. Association for Computational Linguistics.
Cite (Informal):
Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences (Yap et al., Findings 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.findings-emnlp.4.pdf
Code
 BPYap/BERT-WSD
Data
Word Sense Disambiguation: a Unified Evaluation Framework and Empirical Comparison