A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods

Zhihan Zhang, Wenhao Yu, Mengxia Yu, Zhichun Guo, Meng Jiang


Abstract
Multi-task learning (MTL) has become increasingly popular in natural language processing (NLP) because it improves the performance of related tasks by exploiting their commonalities and differences. Nevertheless, it is still not understood very well how multi-task learning can be implemented based on the relatedness of training tasks. In this survey, we review recent advances of multi-task learning methods in NLP, with the aim of summarizing them into two general multi-task training methods based on their task relatedness: (i) joint training and (ii) multi-step training. We present examples in various NLP downstream applications, summarize the task relationships and discuss future directions of this promising topic.
Anthology ID:
2023.eacl-main.66
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
943–956
Language:
URL:
https://aclanthology.org/2023.eacl-main.66
DOI:
10.18653/v1/2023.eacl-main.66
Bibkey:
Cite (ACL):
Zhihan Zhang, Wenhao Yu, Mengxia Yu, Zhichun Guo, and Meng Jiang. 2023. A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 943–956, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods (Zhang et al., EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-main.66.pdf
Video:
 https://aclanthology.org/2023.eacl-main.66.mp4