ZYC at SemEval-2026 Task 5: Application of BERT-based Contextual Embeddings Similarity for WSD

Sunny Zhou, Jordan Youner, Dean Cahill


Abstract
We investigate contextual embedding manipulation for Word Sense Disambiguation (WSD)as part of SemEval-2026 Task 5. We propose four approaches built on BERT-like pretrainedmodels, experimenting with the informativeness of similarity calculations and classificationmethods. We introduce scratch-trained cross-attention mechanisms inspired by GLiNER to compute similarity between definition or synonym representations and the full context. Our best performance achieved 57% accuracy with a Spearman correlation of 0.20. Our results suggest that finetuning strategy and trainng curriculum matter more than pretrained model choice for this novel task, and we identify several directions for future improvement. View our code base at: https://github.com/heliosraz/SemEval52026
Anthology ID:
2026.semeval-1.371
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2960–2966
Language:
URL:
https://aclanthology.org/2026.semeval-1.371/
DOI:
Bibkey:
Cite (ACL):
Sunny Zhou, Jordan Youner, and Dean Cahill. 2026. ZYC at SemEval-2026 Task 5: Application of BERT-based Contextual Embeddings Similarity for WSD. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 2960–2966, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
ZYC at SemEval-2026 Task 5: Application of BERT-based Contextual Embeddings Similarity for WSD (Zhou et al., SemEval 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.semeval-1.371.pdf
Supplementarymaterial:
 2026.semeval-1.371.SupplementaryMaterial.zip