ReAugKD: Retrieval-Augmented Knowledge Distillation For Pre-trained Language Models

ReAugKD: Retrieval-Augmented Knowledge Distillation For Pre-trained Language Models Jianyi Zhang author Aashiq Muhamed author Aditya Anantharaman author Guoyin Wang author Changyou Chen author Kai Zhong author Qingjun Cui author Yi Xu author Belinda Zeng author Trishul Chilimbi author Yiran Chen author 2023-07 text Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) Anna Rogers editor Jordan Boyd-Graber editor Naoaki Okazaki editor Association for Computational Linguistics Toronto, Canada conference publication zhang-etal-2023-reaugkd 10.18653/v1/2023.acl-short.97 https://aclanthology.org/2023.acl-short.97/ 2023-07 1128 1136