Learning to Solve NLP Tasks in an Incremental Number of Languages

Giuseppe Castellucci, Simone Filice, Danilo Croce, Roberto Basili


Abstract
In real scenarios, a multilingual model trained to solve NLP tasks on a set of languages can be required to support new languages over time. Unfortunately, the straightforward retraining on a dataset containing annotated examples for all the languages is both expensive and time-consuming, especially when the number of target languages grows. Moreover, the original annotated material may no longer be available due to storage or business constraints. Re-training only with the new language data will inevitably result in Catastrophic Forgetting of previously acquired knowledge. We propose a Continual Learning strategy that updates a model to support new languages over time, while maintaining consistent results on previously learned languages. We define a Teacher-Student framework where the existing model “teaches” to a student model its knowledge about the languages it supports, while the student is also trained on a new language. We report an experimental evaluation in several tasks including Sentence Classification, Relational Learning and Sequence Labeling.
Anthology ID:
2021.acl-short.106
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Month:
August
Year:
2021
Address:
Online
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
837–847
Language:
URL:
https://aclanthology.org/2021.acl-short.106
DOI:
10.18653/v1/2021.acl-short.106
Bibkey:
Cite (ACL):
Giuseppe Castellucci, Simone Filice, Danilo Croce, and Roberto Basili. 2021. Learning to Solve NLP Tasks in an Incremental Number of Languages. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 837–847, Online. Association for Computational Linguistics.
Cite (Informal):
Learning to Solve NLP Tasks in an Incremental Number of Languages (Castellucci et al., ACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-short.106.pdf
Video:
 https://aclanthology.org/2021.acl-short.106.mp4
Data
PAWS-X