Learning Contextualized Box Embeddings with Prototypical Networks

Kohei Oda, Kiyoaki Shirai, Natthawut Kertkeidkachorn


Abstract
This paper proposes ProtoBox, a novel method to learn contextualized box embeddings. Unlike an ordinary word embedding, which represents a word as a single vector, a box embedding represents the meaning of a word as a box in a high-dimensional space: that is suitable for representing semantic relations between words. In addition, our method aims to obtain a “contextualized” box embedding, which is an abstract representation of a word in a specific context. ProtoBox is based on Prototypical Networks, which is a robust method for classification problems, especially focusing on learning the hypernym–hyponym relation between senses. ProtoBox is evaluated on three tasks: Word Sense Disambiguation (WSD), New Sense Classification (NSC), and Hypernym Identification (HI). Experimental results show that ProtoBox outperforms baselines for the HI task and is comparable for the WSD and NSC tasks.
Anthology ID:
2024.repl4nlp-1.1
Volume:
Proceedings of the 9th Workshop on Representation Learning for NLP (RepL4NLP-2024)
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Chen Zhao, Marius Mosbach, Pepa Atanasova, Seraphina Goldfarb-Tarrent, Peter Hase, Arian Hosseini, Maha Elbayad, Sandro Pezzelle, Maximilian Mozes
Venues:
RepL4NLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–12
Language:
URL:
https://aclanthology.org/2024.repl4nlp-1.1
DOI:
Bibkey:
Cite (ACL):
Kohei Oda, Kiyoaki Shirai, and Natthawut Kertkeidkachorn. 2024. Learning Contextualized Box Embeddings with Prototypical Networks. In Proceedings of the 9th Workshop on Representation Learning for NLP (RepL4NLP-2024), pages 1–12, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Learning Contextualized Box Embeddings with Prototypical Networks (Oda et al., RepL4NLP-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.repl4nlp-1.1.pdf