MTRec: Multi-Task Learning over BERT for News Recommendation

Qiwei Bi, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, Hanfang Yang


Abstract
Existing news recommendation methods usually learn news representations solely based on news titles. To sufficiently utilize other fields of news information such as category and entities, some methods treat each field as an additional feature and combine different feature vectors with attentive pooling. With the adoption of large pre-trained models like BERT in news recommendation, the above way to incorporate multi-field information may encounter challenges: the shallow feature encoding to compress the category and entity information is not compatible with the deep BERT encoding. In this paper, we propose a multi-task method to incorporate the multi-field information into BERT, which improves its news encoding capability. Besides, we modify the gradients of auxiliary tasks based on their gradient conflicts with the main task, which further boosts the model performance. Extensive experiments on the MIND news recommendation benchmark show the effectiveness of our approach.
Anthology ID:
2022.findings-acl.209
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2663–2669
Language:
URL:
https://aclanthology.org/2022.findings-acl.209
DOI:
10.18653/v1/2022.findings-acl.209
Bibkey:
Cite (ACL):
Qiwei Bi, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, and Hanfang Yang. 2022. MTRec: Multi-Task Learning over BERT for News Recommendation. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2663–2669, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
MTRec: Multi-Task Learning over BERT for News Recommendation (Bi et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.209.pdf
Data
MIND