To What Degree Can Language Borders Be Blurred In BERT-based Multilingual Spoken Language Understanding?

Quynh Do, Judith Gaspers, Tobias Roeding, Melanie Bradford


Abstract
This paper addresses the question as to what degree a BERT-based multilingual Spoken Language Understanding (SLU) model can transfer knowledge across languages. Through experiments we will show that, although it works substantially well even on distant language groups, there is still a gap to the ideal multilingual performance. In addition, we propose a novel BERT-based adversarial model architecture to learn language-shared and language-specific representations for multilingual SLU. Our experimental results prove that the proposed model is capable of narrowing the gap to the ideal multilingual performance.
Anthology ID:
2020.coling-main.243
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Donia Scott, Nuria Bel, Chengqing Zong
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
2699–2709
Language:
URL:
https://aclanthology.org/2020.coling-main.243
DOI:
10.18653/v1/2020.coling-main.243
Bibkey:
Cite (ACL):
Quynh Do, Judith Gaspers, Tobias Roeding, and Melanie Bradford. 2020. To What Degree Can Language Borders Be Blurred In BERT-based Multilingual Spoken Language Understanding?. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2699–2709, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
To What Degree Can Language Borders Be Blurred In BERT-based Multilingual Spoken Language Understanding? (Do et al., COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-main.243.pdf