EVALution-MAN: A Chinese Dataset for the Training and Evaluation of DSMs

Liu Hongchao, Karl Neergaard, Enrico Santus, Chu-Ren Huang


Abstract
Distributional semantic models (DSMs) are currently being used in the measurement of word relatedness and word similarity. One shortcoming of DSMs is that they do not provide a principled way to discriminate different semantic relations. Several approaches have been adopted that rely on annotated data either in the training of the model or later in its evaluation. In this paper, we introduce a dataset for training and evaluating DSMs on semantic relations discrimination between words, in Mandarin, Chinese. The construction of the dataset followed EVALution 1.0, which is an English dataset for the training and evaluating of DSMs. The dataset contains 360 relation pairs, distributed in five different semantic relations, including antonymy, synonymy, hypernymy, meronymy and nearsynonymy. All relation pairs were checked manually to estimate their quality. In the 360 word relation pairs, there are 373 relata. They were all extracted and subsequently manually tagged according to their semantic type. The relatas’ frequency was calculated in a combined corpus of Sinica and Chinese Gigaword. To the best of our knowledge, EVALution-MAN is the first of its kind for Mandarin, Chinese.
Anthology ID:
L16-1726
Volume:
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
Month:
May
Year:
2016
Address:
Portorož, Slovenia
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
4583–4587
Language:
URL:
https://aclanthology.org/L16-1726
DOI:
Bibkey:
Cite (ACL):
Liu Hongchao, Karl Neergaard, Enrico Santus, and Chu-Ren Huang. 2016. EVALution-MAN: A Chinese Dataset for the Training and Evaluation of DSMs. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 4583–4587, Portorož, Slovenia. European Language Resources Association (ELRA).
Cite (Informal):
EVALution-MAN: A Chinese Dataset for the Training and Evaluation of DSMs (Hongchao et al., LREC 2016)
Copy Citation:
PDF:
https://aclanthology.org/L16-1726.pdf
Data
EVALution