Rare and Zero-shot Word Sense Disambiguation using Z-Reweighting

Ying Su, Hongming Zhang, Yangqiu Song, Tong Zhang


Abstract
Word sense disambiguation (WSD) is a crucial problem in the natural language processing (NLP) community. Current methods achieve decent performance by utilizing supervised learning and large pre-trained language models. However, the imbalanced training dataset leads to poor performance on rare senses and zero-shot senses. There are more training instances and senses for words with top frequency ranks than those with low frequency ranks in the training dataset. We investigate the statistical relation between word frequency rank and word sense number distribution. Based on the relation, we propose a Z-reweighting method on the word level to adjust the training on the imbalanced dataset. The experiments show that the Z-reweighting strategy achieves performance gain on the standard English all words WSD benchmark. Moreover, the strategy can help models generalize better on rare and zero-shot senses.
Anthology ID:
2022.acl-long.323
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4713–4723
Language:
URL:
https://aclanthology.org/2022.acl-long.323
DOI:
10.18653/v1/2022.acl-long.323
Bibkey:
Cite (ACL):
Ying Su, Hongming Zhang, Yangqiu Song, and Tong Zhang. 2022. Rare and Zero-shot Word Sense Disambiguation using Z-Reweighting. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4713–4723, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Rare and Zero-shot Word Sense Disambiguation using Z-Reweighting (Su et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.323.pdf
Software:
 2022.acl-long.323.software.zip
Code
 suytingwan/wsd-z-reweighting
Data
Word Sense Disambiguation: a Unified Evaluation Framework and Empirical Comparison