SimRelUz: Similarity and Relatedness Scores as a Semantic Evaluation Dataset for Uzbek Language

Ulugbek Salaev, Elmurod Kuriyozov, Carlos Gómez-Rodríguez


Abstract
Semantic relatedness between words is one of the core concepts in natural language processing, thus making semantic evaluation an important task. In this paper, we present a semantic model evaluation dataset: SimRelUz - a collection of similarity and relatedness scores of word pairs for the low-resource Uzbek language. The dataset consists of more than a thousand pairs of words carefully selected based on their morphological features, occurrence frequency, semantic relation, as well as annotated by eleven native Uzbek speakers from different age groups and gender. We also paid attention to the problem of dealing with rare words and out-of-vocabulary words to thoroughly evaluate the robustness of semantic models.
Anthology ID:
2022.sigul-1.26
Volume:
Proceedings of the 1st Annual Meeting of the ELRA/ISCA Special Interest Group on Under-Resourced Languages
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Maite Melero, Sakriani Sakti, Claudia Soria
Venue:
SIGUL
SIG:
SIGUL
Publisher:
European Language Resources Association
Note:
Pages:
199–206
Language:
URL:
https://aclanthology.org/2022.sigul-1.26
DOI:
Bibkey:
Cite (ACL):
Ulugbek Salaev, Elmurod Kuriyozov, and Carlos Gómez-Rodríguez. 2022. SimRelUz: Similarity and Relatedness Scores as a Semantic Evaluation Dataset for Uzbek Language. In Proceedings of the 1st Annual Meeting of the ELRA/ISCA Special Interest Group on Under-Resourced Languages, pages 199–206, Marseille, France. European Language Resources Association.
Cite (Informal):
SimRelUz: Similarity and Relatedness Scores as a Semantic Evaluation Dataset for Uzbek Language (Salaev et al., SIGUL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.sigul-1.26.pdf
Code
 ulugbeksalaev/simreluz
Data
AnlamVer