We introduce Omnilingual MT, a family of multilingual translation (MT) systems covering more than 1,600 languages—the broadest benchmarked coverage of any MT system to date. We make contributions in several key research questions in the field: strategies for the use of monolingual data, curating seed training data, tokenization and suitable architectures, the competitiveness of small specialized MT models as opposed to general purpose large models, and evaluation protocols. Our data strategy combines large-scale multilingual corpora with backtranslation for 2,000+ languages and MeDLEy, a grammatically-diverse seed dataset for 109 low-resource languages. We explore two complementary architectures: OMT-LLAMA, a decoder-only model (1B/3B/8B) built on LLAMA3 with multilingual continual pretraining, and OMT-NLLB, a 3B encoder-decoder model with a novel three-stage training strategy exploiting non-parallel data via autoencoding. For evaluation, we extend the BOUQuET benchmark to 275 languages, develop Met-BOUQuET with human annotations of machine quality in 161 directions, and BLASER 3, a reference-free quality estimation metric. Our specialized 1B–8B models match or exceed 70B general-purpose LLMs, demonstrating that specialization is a more reliable path to multilingual translation than scale alone. Omnilingual MT achieves “passable” translation quality as judged by human annotators for over 440 languages, doubling it over the previous state of the field at ∼200 languages, and translates in zero-shot mode into ∼1,200 languages versus 300–400 for the baselines. With Omnilingual MT, we showcase a path encompassing the full pipeline of data and system development towards designing models that are multilingual from the onset.

In real-world systems, an important requirement for model updates is to avoid regressions in user experience caused by flips of previously correct classifications to incorrect ones. Multiple techniques for that have been proposed in the recent literature. In this paper, we apply one such technique, focal distillation, to model updates in a goal-oriented dialog system and assess its usefulness in practice. In particular, we evaluate its effectiveness for key language understanding tasks, including sentence classification and sequence labeling tasks, we further assess its effect when applied to repeated model updates over time, and test its compatibility with mislabeled data. Our experiments on a public benchmark and data from a deployed dialog system demonstrate that focal distillation can substantially reduce regressions, at only minor drops in accuracy, and that it further outperforms naive supervised training in challenging mislabeled data and label expansion settings.

Datasets used to train deep learning models in industrial settings often exhibit skewed distributions with some samples repeated a large number of times. This paper presents a simple yet effective solution to reduce the increased burden of repeated computation on redundant datasets. Our approach eliminates duplicates at the batch level, without altering the data distribution observed by the model, making it model-agnostic and easy to implement as a plug-and-play module. We also provide a mathematical expression to estimate the reduction in training time that our approach provides. Through empirical evidence, we show that our approach significantly reduces training times on various models across datasets with varying redundancy factors, without impacting their performance on the Named Entity Recognition task, both on publicly available datasets and in real industrial settings. In the latter, the approach speeds training by up to 87%, and by 46% on average, with a drop in model performance of 0.2% relative at worst. We finally release a modular and reusable codebase to further advance research in this area.