When does deep multi-task learning work for loosely related document classification tasks?

Emma Kerinec, Chloé Braud, Anders Søgaard


Abstract
This work aims to contribute to our understanding of when multi-task learning through parameter sharing in deep neural networks leads to improvements over single-task learning. We focus on the setting of learning from loosely related tasks, for which no theoretical guarantees exist. We therefore approach the question empirically, studying which properties of datasets and single-task learning characteristics correlate with improvements from multi-task learning. We are the first to study this in a text classification setting and across more than 500 different task pairs.
Anthology ID:
W18-5401
Volume:
Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
Month:
November
Year:
2018
Address:
Brussels, Belgium
Editors:
Tal Linzen, Grzegorz Chrupała, Afra Alishahi
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–8
Language:
URL:
https://aclanthology.org/W18-5401
DOI:
10.18653/v1/W18-5401
Bibkey:
Cite (ACL):
Emma Kerinec, Chloé Braud, and Anders Søgaard. 2018. When does deep multi-task learning work for loosely related document classification tasks?. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 1–8, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
When does deep multi-task learning work for loosely related document classification tasks? (Kerinec et al., EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-5401.pdf