Synthetic data augmentation, especially back-translation with Large Language Models (LLMs), has become a standard strategy for low-resource Neural Machine Translation (NMT). However, jointly training on synthetic and authentic data may suffer from conflicts. We propose GRADient Rectification based on Authentic Data (GRAD), an asymmetric gradient-level framework that treats authentic parallel data as an Authentic Anchor, and rectifies conflicting synthetic updates through orthogonal projection while preserving non-conflicting components. Furthermore, we leverage gradients of multiple authentic batch for comparison and use the most similar one for gradient rectification to alleviate information loss with the synthetic data. Experiments on six low-resource pairs across three LLM families (Qwen, Gemma, Llama) and different scales (4B/8B) show that GRAD consistently outperforms supervised fine-tuning, back-translation, and multi-stage baselines. Notably, on the lowest-resource En-Kk pair, GRAD achieves the highest COMET (83.65) among all evaluated systems. Further analysis shows that synthetic-induced gradient conflicts are localized, and parameter-wise rectification substantially outperforms global-wise rectification.

Self-distillation distills the deeper sub-networks to the shallower sub-networks without using an extra teacher model, and has been proven effective in improving the performance of a series of computer vision tasks. In this paper, we study the representation-based self-distillation methods for Neural Machine Translation (NMT) considering the efficiency issue with a large vocabulary. We present a rank-order augmented Pearson correlation loss and an iterative distillation method to prevent the discrepancy of predictions between the student and a stronger teacher from disturbing the training. To prevent the teacher from misleading the student’s learning, we utilize a warm-up strategy and present a gradient adaption method to scale down or zero the Knowledge Distillation (KD) gradients which are opposite to the translation. Experiments show that our method can lead to significant improvements over the strong Transformer baseline on low/middle/high-resource tasks, obtaining comparable performance to previous MT KD studies without pre-training a teacher. Deeper Transformer experiments show that our method can lead to comparable or better performance with fewer layers.