R-grams: Unsupervised Learning of Semantic Units in Natural Language

Amaru Cuba Gyllensten, Ariel Ekgren, Magnus Sahlgren


Abstract
This paper investigates data-driven segmentation using Re-Pair or Byte Pair Encoding-techniques. In contrast to previous work which has primarily been focused on subword units for machine translation, we are interested in the general properties of such segments above the word level. We call these segments r-grams, and discuss their properties and the effect they have on the token frequency distribution. The proposed approach is evaluated by demonstrating its viability in embedding techniques, both in monolingual and multilingual test settings. We also provide a number of qualitative examples of the proposed methodology, demonstrating its viability as a language-invariant segmentation procedure.
Anthology ID:
W19-0607
Volume:
Proceedings of the 13th International Conference on Computational Semantics - Student Papers
Month:
May
Year:
2019
Address:
Gothenburg, Sweden
Editors:
Simon Dobnik, Stergios Chatzikyriakidis, Vera Demberg, Kathrein Abu Kwaik, Vladislav Maraev
Venue:
IWCS
SIG:
SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
52–62
Language:
URL:
https://aclanthology.org/W19-0607
DOI:
10.18653/v1/W19-0607
Bibkey:
Cite (ACL):
Amaru Cuba Gyllensten, Ariel Ekgren, and Magnus Sahlgren. 2019. R-grams: Unsupervised Learning of Semantic Units in Natural Language. In Proceedings of the 13th International Conference on Computational Semantics - Student Papers, pages 52–62, Gothenburg, Sweden. Association for Computational Linguistics.
Cite (Informal):
R-grams: Unsupervised Learning of Semantic Units in Natural Language (Gyllensten et al., IWCS 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-0607.pdf