Polar Embedding

Ran Iwamoto, Ryosuke Kohita, Akifumi Wachi


Abstract
Hierarchical relationships are invaluable information for many natural language processing (NLP) tasks. Distributional representation has become a fundamental approach for encoding word relationships, particularly embeddings in hyperbolic space showed great performance in representing hierarchies by taking advantage of their spatial properties. However, most machine learning systems do not suppose to use in such complex non-Euclidean geometries. To achieve hierarchy representations in commonly used Euclidean space, we propose Polar Embedding that learns word embeddings with the polar coordinate system. Utilizing characteristics of polar coordinates, the hierarchy of words is expressed with two independent variables: radius (generality) and angles (similarity), and their variables are optimized separately. Polar embedding shows word hierarchies explicitly and allows us to use beneficial resources such as word frequencies or word generality annotations for computing radiuses. We introduce an optimization method for learning angles in limited ranges of polar coordinates, which combining a loss function controlling gradient and distribution uniformization. Experimental results on hypernymy datasets indicate that our approach outperforms other embeddings in low-dimensional Euclidean space and competitively performs even with hyperbolic embeddings, which possess a geometric advantage.
Anthology ID:
2021.conll-1.37
Volume:
Proceedings of the 25th Conference on Computational Natural Language Learning
Month:
November
Year:
2021
Address:
Online
Editors:
Arianna Bisazza, Omri Abend
Venue:
CoNLL
SIG:
SIGNLL
Publisher:
Association for Computational Linguistics
Note:
Pages:
470–480
Language:
URL:
https://aclanthology.org/2021.conll-1.37
DOI:
10.18653/v1/2021.conll-1.37
Bibkey:
Cite (ACL):
Ran Iwamoto, Ryosuke Kohita, and Akifumi Wachi. 2021. Polar Embedding. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 470–480, Online. Association for Computational Linguistics.
Cite (Informal):
Polar Embedding (Iwamoto et al., CoNLL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.conll-1.37.pdf
Software:
 2021.conll-1.37.Software.zip
Video:
 https://aclanthology.org/2021.conll-1.37.mp4