Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation

Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation Yuhang Zhou author Jing Zhu author Paiheng Xu author Xiaoyu Liu author Xiyao Wang author Danai Koutra author Wei Ai author Furong Huang author 2024-11 text Findings of the Association for Computational Linguistics: EMNLP 2024 Yaser Al-Onaizan editor Mohit Bansal editor Yun-Nung Chen editor Association for Computational Linguistics Miami, Florida, USA conference publication zhou-etal-2024-multi 10.18653/v1/2024.findings-emnlp.190 https://aclanthology.org/2024.findings-emnlp.190/ 2024-11 3315 3333