Uncertainty Regularized Multi-Task Learning

Kourosh Meshgi, Maryam Sadat Mirzaei, Satoshi Sekine


Abstract
By sharing parameters and providing task-independent shared features, multi-task deep neural networks are considered one of the most interesting ways for parallel learning from different tasks and domains. However, fine-tuning on one task may compromise the performance of other tasks or restrict the generalization of the shared learned features. To address this issue, we propose to use task uncertainty to gauge the effect of the shared feature changes on other tasks and prevent the model from overfitting or over-generalizing. We conducted an experiment on 16 text classification tasks, and findings showed that the proposed method consistently improves the performance of the baseline, facilitates the knowledge transfer of learned features to unseen data, and provides explicit control over the generalization of the shared model.
Anthology ID:
2022.wassa-1.8
Volume:
Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Jeremy Barnes, Orphée De Clercq, Valentin Barriere, Shabnam Tafreshi, Sawsan Alqahtani, João Sedoc, Roman Klinger, Alexandra Balahur
Venue:
WASSA
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
78–88
Language:
URL:
https://aclanthology.org/2022.wassa-1.8
DOI:
10.18653/v1/2022.wassa-1.8
Bibkey:
Cite (ACL):
Kourosh Meshgi, Maryam Sadat Mirzaei, and Satoshi Sekine. 2022. Uncertainty Regularized Multi-Task Learning. In Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, pages 78–88, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Uncertainty Regularized Multi-Task Learning (Meshgi et al., WASSA 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.wassa-1.8.pdf
Video:
 https://aclanthology.org/2022.wassa-1.8.mp4
Data
IMDb Movie Reviews