Investigating Sub-Word Embedding Strategies for the Morphologically Rich and Free Phrase-Order Hungarian

Bálint Döbrössy, Márton Makrai, Balázs Tarján, György Szaszák


Abstract
For morphologically rich languages, word embeddings provide less consistent semantic representations due to higher variance in word forms. Moreover, these languages often allow for less constrained word order, which further increases variance. For the highly agglutinative Hungarian, semantic accuracy of word embeddings measured on word analogy tasks drops by 50-75% compared to English. We observed that embeddings learn morphosyntax quite well instead. Therefore, we explore and evaluate several sub-word unit based embedding strategies – character n-grams, lemmatization provided by an NLP-pipeline, and segments obtained in unsupervised learning (morfessor) – to boost semantic consistency in Hungarian word vectors. The effect of changing embedding dimension and context window size have also been considered. Morphological analysis based lemmatization was found to be the best strategy to improve embeddings’ semantic accuracy, whereas adding character n-grams was found consistently counterproductive in this regard.
Anthology ID:
W19-4321
Volume:
Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
Month:
August
Year:
2019
Address:
Florence, Italy
Venues:
ACL | RepL4NLP | WS
SIG:
SIGREP
Publisher:
Association for Computational Linguistics
Note:
Pages:
187–193
Language:
URL:
https://aclanthology.org/W19-4321
DOI:
10.18653/v1/W19-4321
Bibkey:
Cite (ACL):
Bálint Döbrössy, Márton Makrai, Balázs Tarján, and György Szaszák. 2019. Investigating Sub-Word Embedding Strategies for the Morphologically Rich and Free Phrase-Order Hungarian. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 187–193, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Investigating Sub-Word Embedding Strategies for the Morphologically Rich and Free Phrase-Order Hungarian (Döbrössy et al., 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-4321.pdf