Warren at SemEval-2020 Task 4: ALBERT and Multi-Task Learning for Commonsense Validation

Yuhang Wu, Hao Wu


Abstract
This paper describes our system in subtask A of SemEval 2020 Shared Task 4. We propose a reinforcement learning model based on MTL(Multi-Task Learning) to enhance the prediction ability of commonsense validation. The experimental results demonstrate that our system outperforms the single-task text classification model. We combine MTL and ALBERT pretrain model to achieve an accuracy of 0.904 and our model is ranked 16th on the final leader board of the competition among the 45 teams.
Anthology ID:
2020.semeval-1.79
Volume:
Proceedings of the Fourteenth Workshop on Semantic Evaluation
Month:
December
Year:
2020
Address:
Barcelona (online)
Editors:
Aurelie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May, Ekaterina Shutova
Venue:
SemEval
SIG:
SIGLEX
Publisher:
International Committee for Computational Linguistics
Note:
Pages:
620–625
Language:
URL:
https://aclanthology.org/2020.semeval-1.79
DOI:
10.18653/v1/2020.semeval-1.79
Bibkey:
Cite (ACL):
Yuhang Wu and Hao Wu. 2020. Warren at SemEval-2020 Task 4: ALBERT and Multi-Task Learning for Commonsense Validation. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 620–625, Barcelona (online). International Committee for Computational Linguistics.
Cite (Informal):
Warren at SemEval-2020 Task 4: ALBERT and Multi-Task Learning for Commonsense Validation (Wu & Wu, SemEval 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.semeval-1.79.pdf