GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model

GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model Shicheng Tan author Weng Lam Tam author Yuanchun Wang author Wenwen Gong author Shu Zhao author Peng Zhang author Jie Tang author 2023-07 text Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track) Sunayana Sitaram editor Beata Beigman Klebanov editor Jason D Williams editor Association for Computational Linguistics Toronto, Canada conference publication tan-etal-2023-gkd 10.18653/v1/2023.acl-industry.15 https://aclanthology.org/2023.acl-industry.15/ 2023-07 134 148