Polish evaluation dataset for compositional distributional semantics models

Alina Wróblewska, Katarzyna Krasnowska-Kieraś


Abstract
The paper presents a procedure of building an evaluation dataset. for the validation of compositional distributional semantics models estimated for languages other than English. The procedure generally builds on steps designed to assemble the SICK corpus, which contains pairs of English sentences annotated for semantic relatedness and entailment, because we aim at building a comparable dataset. However, the implementation of particular building steps significantly differs from the original SICK design assumptions, which is caused by both lack of necessary extraneous resources for an investigated language and the need for language-specific transformation rules. The designed procedure is verified on Polish, a fusional language with a relatively free word order, and contributes to building a Polish evaluation dataset. The resource consists of 10K sentence pairs which are human-annotated for semantic relatedness and entailment. The dataset may be used for the evaluation of compositional distributional semantics models of Polish.
Anthology ID:
P17-1073
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Editors:
Regina Barzilay, Min-Yen Kan
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
784–792
Language:
URL:
https://aclanthology.org/P17-1073
DOI:
10.18653/v1/P17-1073
Bibkey:
Cite (ACL):
Alina Wróblewska and Katarzyna Krasnowska-Kieraś. 2017. Polish evaluation dataset for compositional distributional semantics models. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 784–792, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Polish evaluation dataset for compositional distributional semantics models (Wróblewska & Krasnowska-Kieraś, ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-1073.pdf
Video:
 https://aclanthology.org/P17-1073.mp4
Data
Polish CDSCorpus