Fengzhao Fan

Author directory

2026

This paper presents Xiaoyu-MT, our submission system for the WMT 2026 Chinese-Southeast Asian Multilingual MT Task, focusing on bidirectional translation between Chinese and seven Southeast Asian languages. To address limited parallel resources and imbalanced multilingual data distributions, we select Gemma-4-12B-it as the base model and construct multilingual training data by combining official WMT resources with additional monolingual and bilingual corpora. Xiaoyu-MT adopts a multi-stage training pipeline consisting of Continual Pre-Training (CPT), Supervised Fine-Tuning (SFT), and Direct Preference Optimization (DPO). During SFT, we introduce a language-aware Prompt encoding mechanism to explicitly model source language, target language, and their directed translation relationships through structured Prompt representations and Prompt compression. DPO further optimizes translation preferences using preference data constructed from reference and model-generated translations. Official evaluation results on WMT 2026 show that Xiaoyu-MT achieves an overall BLEU score of 34.73, a COMET score of 81.87, and a Final Result score of 61.80, demonstrating its effectiveness for multilingual and low-resource machine translation scenarios.
Large Language Models (LLMs) enable a single model to support translation across multiple language directions simultaneously, eliminating the need to train a separate model for each direction and substantially improving the performance of multilingual machine translation. However, during multilingual translation instruction tuning, parameter interference among different languages significantly limits the improvement of translation performance and may even cause performance degradation for some languages. To address this issue, we propose MeRP, a multi-expert routing prompt encoding strategy for parameter-efficient fine-tuning (PEFT) in multilingual machine translation tasks. This method encodes representations of source languages through an attention mechanism and dynamically selects expert groups according to translation directions, enabling both the sharing and disentanglement of multilingual semantic representations. As a result, it preserves LLM generalization while reducing training time and improving translation performance. We conduct experiments on five language pairs. The results show that, compared with the baseline models, our method updates only 0.06% of the total model parameters during training while achieving better multilingual translation performance. Notably, our MeRP reduces training time by 46.75% and achieves average gains of +0.62 BLEU points and +0.34 COMET-22 points over the arithmetic mean of the four PEFT baselines.Code is available at https://github.com/wangzhenhan93/MeRP.