Recent advancements in Large Language Models (LLMs) have shown remarkable capabilities in multilingual contexts. However, previous studies suggest that multilingual LLMs primarily rely on the rich semantic knowledge of high-resource (i.e., English) representations to conduct tasks across languages. To mitigate the performance gap arising from resource imbalances, we propose Steering via Language-Specific Neurons, a training-free method that aligns non-English representations with the English manifold intervening on language-specific neuron activations via linear interpolation toward English activation prototypes. Experimental results show that the proposed method generally improves downstream task performance with minimal computational cost, yielding performance gains for low-resource scenarios where limited data often constrains knowledge coverage. Our findings suggest that the fundamental limitations of low-resource languages can be effectively alleviated by leveraging the robust representation space of high-resource languages, offering a promising direction for more balanced multilingual LLMs.

Distilling the tool-use capabilities of large language models (LLMs) into small language models (SLMs) is essential for their practical application. The predominant approach, supervised fine-tuning (SFT), is an off-policy distillation method that suffers from poor out-of-domain (OOD) generalization because it rigidly aligns with static teacher trajectories. While reinforcement learning (RL) offers an alternative, the capacity limitations of SLMs pose a severe dilemma: sparse outcome rewards provide insufficient guidance, whereas strict trajectory matching imposes overly restrictive constraints. To bridge this capacity-driven gap, we propose MENTOR, an on-policy distillation framework that introduces a flexible yet process-aware reward structure. Instead of enforcing rigid replication, MENTOR uses the teacher’s reference to guide tool-use behavior, balancing behavioral alignment with downstream performance. Extensive experiments on controlled executable-tool benchmarks demonstrate that MENTOR improves OOD tool-use performance compared to SFT and strict RL baselines. Our findings suggest that within verifiable tool-use environments, flexible tool-use alignment offers a more effective approach than strict trajectory replication for developing adaptable small models.

Multilingual large language models (mLLMs) contain a small fraction of feed-forward neurons that are sensitive to particular languages, commonly termed language-specific neurons. Existing work measures language specificity using the entropy of each neuron’s language-wise probabilities of being active, where a neuron is considered active when its activation value is positive. However, this approach may not fully capture the multilingual nature of mLLMs, where language representations are distributional and mutually related. We propose Distribution-aware Language Neuron selection, which leverages pairwise relationships between per-language activation distributions over the full activation range, including negative values. Specifically, we quantify each neuron’s language specificity by clustering languages using pairwise overlap coefficients between their activation distributions. Across two mLLMs and two held-out corpora, our identifier more effectively isolates language-specific causal effects, yielding up to 4.9 times higher on-target language damage per neuron while preserving off-target language performance.