Shuting Jiang

Author directory

2026

This paper presents Xiaoyu-MT, our submission system for the WMT 2026 Chinese-Southeast Asian Multilingual MT Task, focusing on bidirectional translation between Chinese and seven Southeast Asian languages. To address limited parallel resources and imbalanced multilingual data distributions, we select Gemma-4-12B-it as the base model and construct multilingual training data by combining official WMT resources with additional monolingual and bilingual corpora. Xiaoyu-MT adopts a multi-stage training pipeline consisting of Continual Pre-Training (CPT), Supervised Fine-Tuning (SFT), and Direct Preference Optimization (DPO). During SFT, we introduce a language-aware Prompt encoding mechanism to explicitly model source language, target language, and their directed translation relationships through structured Prompt representations and Prompt compression. DPO further optimizes translation preferences using preference data constructed from reference and model-generated translations. Official evaluation results on WMT 2026 show that Xiaoyu-MT achieves an overall BLEU score of 34.73, a COMET score of 81.87, and a Final Result score of 61.80, demonstrating its effectiveness for multilingual and low-resource machine translation scenarios.
Although Large Language Models (LLMs) have achieved remarkable success in Machine Translation (MT), a significant performance gap persists between high-resource and low-resource languages due to imbalanced pre-training data. In this paper, we first investigate the internal mechanisms driving this performance disparity from a layer-wise perspective.We propose a metric termed Activation Disparity (𝛥 R) to quantify the activation divergence between high- and low-resource MT. Based on this metric, we distinguish between Task-Adaptive Layers (TAL, 𝛥 R > 0) that encode task-specific signals and Legacy-Inert Layers (LIL, 𝛥 R < 0) dominated by pre-trained bias. Leveraging this finding, we propose the Layer-aware Dual-directional Modulation (LaDM). Integrated with Low-Rank Adaptation (LoRA), LaDM employs a sparse strategy to bidirectionally modulate optimization dynamics. Specifically, it amplifies contributions from TAL to accelerate feature consolidation while inhibiting LIL to dampen misaligned legacy biases. Extensive experiments on Chinese-to-seven low-resource language translation using Llama-3.1, Qwen2.5, and Gemma-2 demonstrate that LaDM significantly outperforms standard LoRA fine-tuning, achieving an average improvement of 1.73 spBLEU.Code is available at https://github.com/zzssqqq/LaDM.
Reinforcement learning (RL) excels in reasoning tasks with verifiable rewards, while its adaptation to machine translation (MT) remains challenging due to the lack of unique reward signals under multiple valid translations. Existing RL approaches for MT face either fixed references in supervised settings or the production of homogeneous references leading to mode collapse in unsupervised settings. Both limitations arise from ignoring entropy dynamics in RL-based MT. The core challenge is leveraging entropy for supervision construction and self-evolution. In this paper, we propose an Entropy-Driven Unsupervised RL for MT. Our framework integrates entropy-guided sampling for exploration, confidence-weighted label generation to transcend majority-voting bias, and uncertainty-aware optimization to prioritize high-entropy tokens. These mechanisms allow reward signals to co-evolve with model proficiency beyond fixed references. Experiments across multiple language pairs show our method outperforms supervised and unsupervised baselines by +0.63 and +2.52 average points, respectively. Our code is available at https://github.com/fortunatekiss/URLMT.

2024

Large language models (LLMs) have demonstrated remarkable capabilities in comprehensively handling various types of natural language processing (NLP) tasks. However, there are significant differences in the knowledge and abilities required for different tasks. Therefore, it is important to understand whether the same LLM processes different tasks in the same way. Are there specific neurons in a LLM for different tasks? Inspired by neuroscience, this paper pioneers the exploration of whether distinct neurons are activated when a LLM handles different tasks. Compared with current research exploring the neurons of language and knowledge, task-specific neurons present a greater challenge due to their abstractness, diversity, and complexity. To address these challenges, this paper proposes a method for task-specific neuron localization based on Causal Gradient Variation with Special Tokens (CGVST). CGVST identifies task-specific neurons by concentrating on the most significant tokens during task processing, thereby eliminating redundant tokens and minimizing interference from non-essential neurons. Compared to traditional neuron localization methods, our approach can more effectively identify task-specific neurons. We conduct experiments across eight different public tasks. Experiments involving the inhibition and amplification of identified neurons demonstrate that our method can accurately locate task-specific neurons.