TRAVIS at PARSEME Shared Task 2020: How good is (m)BERT at seeing the unseen?

Murathan Kurfalı


Abstract
This paper describes the TRAVIS system built for the PARSEME Shared Task 2020 on semi-supervised identification of verbal multiword expressions. TRAVIS is a fully feature-independent model, relying only on the contextual embeddings. We have participated with two variants of TRAVIS, TRAVIS-multi and TRAVIS-mono, where the former employs multilingual contextual embeddings and the latter uses monolingual ones. Our systems are ranked second and third among seven submissions in the open track, respectively. Despite the strong performance of multilingual contextual embeddings across all languages, the results show that language-specific contextual embeddings have better generalization capabilities.
Anthology ID:
2020.mwe-1.18
Volume:
Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons
Month:
December
Year:
2020
Address:
online
Editors:
Stella Markantonatou, John McCrae, Jelena Mitrović, Carole Tiberius, Carlos Ramisch, Ashwini Vaidya, Petya Osenova, Agata Savary
Venue:
MWE
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
136–141
Language:
URL:
https://aclanthology.org/2020.mwe-1.18
DOI:
Bibkey:
Cite (ACL):
Murathan Kurfalı. 2020. TRAVIS at PARSEME Shared Task 2020: How good is (m)BERT at seeing the unseen?. In Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pages 136–141, online. Association for Computational Linguistics.
Cite (Informal):
TRAVIS at PARSEME Shared Task 2020: How good is (m)BERT at seeing the unseen? (Kurfalı, MWE 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.mwe-1.18.pdf
Code
 MurathanKurfali/travis