Evfrosiniya Zerminova
Author directory2026
Findings of the WMT26 Multilingual Instruction Shared Task: Small Models Are Not Yet Polyglots
Patricia Schmidtova | Ekaterina Artemova | Seth Aycock | Niyati Bafna | Shobhit Banga | Manmeet Kaur | Tom Kocmi | Philipp Koehn | Danni Liu | Nam Luu | Sara Papi | Beatrice Savoldi | Mariya Shmatova | Hanuman Sidh | Evfrosiniya Zerminova | Vilem Zouhar | Maike Züfle | Pinzhen Chen
Proceedings of the Eleventh Conference on Machine Translation
Patricia Schmidtova | Ekaterina Artemova | Seth Aycock | Niyati Bafna | Shobhit Banga | Manmeet Kaur | Tom Kocmi | Philipp Koehn | Danni Liu | Nam Luu | Sara Papi | Beatrice Savoldi | Mariya Shmatova | Hanuman Sidh | Evfrosiniya Zerminova | Vilem Zouhar | Maike Züfle | Pinzhen Chen
Proceedings of the Eleventh Conference on Machine Translation
We present the findings of the WMT26 Multilingual Instruction Shared Task (MIST), evaluating small (=10B parameter) open-weight language models across 24 languages on three sub-tasks: context-based QA, cross-lingual summarization, and open-ended generation. Submissions from eight teams explore fine-tuning, knowledge distillation, and prompt engineering. Our findings demonstrate that small models are not yet polyglots: while leading systems perform well on high-resource languages, performance drops sharply on low-resource varieties, and comprehension degrades significantly when transferring across non-English language pairs. In open-ended generation, human evaluation and automatic rule-based verification exhibit strong overall rank correlation (0.79) but diverge locally at the top: human evaluators favor distilled pipelines with superior fluency and cross-lingual equity, whereas automatic verifiers reward concise prompt-engineered systems. Furthermore, multi-stage pipelines suffer from prompt-language inertia on cross-lingual instructions, and summarization exhibits severe English leakage unless penalized by language gating. Finally, while baseline fluency is largely achieved by modern small models, multi-constraint instruction following and unanswerable question refusal remain the primary performance bottlenecks. We release all test sets, system outputs, human judgments, and evaluation code under a permissive license.