Mei Gao
Author directory2026
Divide, Conquer, and Distill: Unified Sample Selection and LLM Distillation for Noisy Long-Tailed Learning
Bo Yuan | Yulin Chen | Jie Cao | Mei Gao
Findings of the Association for Computational Linguistics: EMNLP 2026
Bo Yuan | Yulin Chen | Jie Cao | Mei Gao
Findings of the Association for Computational Linguistics: EMNLP 2026
There is a rising interest in further exploring learning with noisy labels on textual data. Existing solutions often assume the ground truth classes of the training data are balanced, overlooking the long-tailed distribution prevalently encountered in real-world data, which can significantly harm the effect of denoising strategies. Meanwhile, misidentifying clean samples further compromises the model’s learning of long-tailed data. To tackle this issue, we propose a novel knowledge distillation framework for learning with noisy labels under long-tailed distribution (LNL-LT). Specifically, we first introduce a divide-and-conquer sample selection strategy to perform head-tail class division and better separate clean samples from noisy samples within these classes, especially for the tail classes. Then, we employ LLM knowledge distillation to generate pseudo-labels and new samples for tail classes, enhancing pseudo-label quality while mitigating the effects of sample scarcity caused by the long-tailed distribution. Experimental results on both synthetic and real-world datasets demonstrate that our method surpasses current state-of-the-art methods.