SemEval-2020 Task 2: Predicting Multilingual and Cross-Lingual (Graded) Lexical Entailment

Goran Glavaš, Ivan Vulić, Anna Korhonen, Simone Paolo Ponzetto


Abstract
Lexical entailment (LE) is a fundamental asymmetric lexico-semantic relation, supporting the hierarchies in lexical resources (e.g., WordNet, ConceptNet) and applications like natural language inference and taxonomy induction. Multilingual and cross-lingual NLP applications warrant models for LE detection that go beyond language boundaries. As part of SemEval 2020, we carried out a shared task (Task 2) on multilingual and cross-lingual LE. The shared task spans three dimensions: (1) monolingual vs. cross-lingual LE, (2) binary vs. graded LE, and (3) a set of 6 diverse languages (and 15 corresponding language pairs). We offered two different evaluation tracks: (a) Dist: for unsupervised, fully distributional models that capture LE solely on the basis of unannotated corpora, and (b) Any: for externally informed models, allowed to leverage any resources, including lexico-semantic networks (e.g., WordNet or BabelNet). In the Any track, we recieved runs that push state-of-the-art across all languages and language pairs, for both binary LE detection and graded LE prediction.
Anthology ID:
2020.semeval-1.2
Volume:
Proceedings of the Fourteenth Workshop on Semantic Evaluation
Month:
December
Year:
2020
Address:
Barcelona (online)
Editors:
Aurelie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May, Ekaterina Shutova
Venue:
SemEval
SIG:
SIGLEX
Publisher:
International Committee for Computational Linguistics
Note:
Pages:
24–35
Language:
URL:
https://aclanthology.org/2020.semeval-1.2
DOI:
10.18653/v1/2020.semeval-1.2
Bibkey:
Cite (ACL):
Goran Glavaš, Ivan Vulić, Anna Korhonen, and Simone Paolo Ponzetto. 2020. SemEval-2020 Task 2: Predicting Multilingual and Cross-Lingual (Graded) Lexical Entailment. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 24–35, Barcelona (online). International Committee for Computational Linguistics.
Cite (Informal):
SemEval-2020 Task 2: Predicting Multilingual and Cross-Lingual (Graded) Lexical Entailment (Glavaš et al., SemEval 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.semeval-1.2.pdf
Data
ConceptNetHyperLex