Effective communication in asylum reception settings requires reliable machine translation (MT) across many languages, including low-resource ones. Using data from the ANON project, we compare retrieval-augmented LLM translation with adaptive Neural MT across 14 target languages with varying resource levels. Working with a very small translation memory of only 358 sentences, we evaluate fuzzy match (FM) augmentation as an in-context learning strategy for open-source and commercial LLMs and benchmark these against ModernMT with and without domain adaptation. In the LLM setting, FM-based example selection consistently outperforms random selection and zero-shot prompting, with the largest gains for low-resource languages. Adaptive NMT retains an overall advantage, although Gemini~Pro approaches its performance and outperforms it on 6 of 14 languages, highlighting a trade-off between translation quality and data sovereignty in privacy-sensitive contexts. These findings show that FM augmentation remains effective under severe data constraints and emphasise the importance of language-specific evaluation in multilingual MT.

Large Language Models (LLMs) have recently demonstrated strong performance in machine translation (MT). However, most prior work focuses on improving or benchmarking translation quality, offering limited insight into when and why LLM-based translation fails. In this work, we systematically analyze failure modes of LLMs in MT by evaluating 15 models, including four reasoning LLMs, across 22 language pairs (LPs) with varying resource levels. We find that non-English-centric LPs consistently yield lower COMET scores than English-centric pairs. To investigate the underlying causes, we introduce Token Activation Rate (TAR), a metric that captures how effectively a model utilizes language-specific tokens in its vocabulary during generation. We validate TAR as a proxy for language representation using models with known language distributions in the training data, and show that lower TAR is strongly associated with poorer translation performance. Furthermore, reasoning LLMs tend to generate more tokens when translating into low-TAR languages, suggesting a compensatory mechanism, although its impact on translation quality varies across models. Overall, our findings emphasize the importance of token-level dynamics in understanding MT performance of LLMs.

Timely and reliable multilingual communication is critical during natural and human-induced disasters, but developing effective solutions for crisis communication is limited by the scarcity of curated parallel data. We propose a domain-adaptive pipeline that expands a small reference corpus, by retrieving and filtering data from general corpora. We use the resulting dataset to fine-tune a small language model for crisis-domain translation and then apply preference optimization to bias outputs toward CEFR A2-level English. Automatic and human evaluation shows that this approach improves readability, while maintaining strong adequacy. Our results indicate that simplified English, combined with domain adaptation, can function as a practical lingua franca for emergency communication when full multilingual coverage is not feasible.

Generative AI tools, such as ChatGPT, are applied to a wide range of languagerelated tasks, including translation. Despite their current popularity among users and researchers and the impressive results obtained on several benchmarks (Kocmi et al., 2024a; Deutsch et al., 2025), their potential side-effects on languages and translations are still understudied (Vanmassenhove, 2025). The paradigm shift from Machine Translation (MT) to Generative AI Translation (GAIT) likely calls for a reconsideration of our assessment and evaluation metrics and practices. In this work, we focus on GAIT by analyzing translations from four multilingual large language models (MLLMs), mBART, Jamba-1.5-large, GPT 4o and DeepSeek R1 applied to three different domains (news, literature and poetry) for the English-Dutch language pair. Focusing on metrics related to lexical and textual diversity, we find that while GAIT text for literature if of significantly high lexical and grammatical richness, that is not the case for news and poetry. We also assess the homogeneity of AI-generated text through a set of clustering and classification experiments. In addition to a clear separation between human- and AI-generated content, our results indicate that GAIT output is more homogeneous among MLLMs.

We propose an LLM-as-a-Jury framework for determining machine translation publishability, aggregating judgments from multiple large language models via logistic regression rather than relying on a single judge. Publishability is defined as the absence of major or critical errors—those that render a translation unsuitable for public release without human post-editing. We compare three evaluation frameworks: a generic Edit Effort Estimation (EEE) prompt based on lexical accuracy, grammatical correctness and semantic coherence, a generic Linguistic Quality Assurance (LQA) prompt based on the MQM error taxonomy, and a purpose-built Publishability prompt optimized via DSPy and augmented with domain-specific fine-tuning. Experiments across three domains and nine language pairs show that (i) the jury ensemble matches or outperforms the best individual juror in nearly every condition, (ii) EEE and LQA juries are competitive with and occasionally exceed the Publishability jury on macro-F1, (iii) the Publishability framework offers stronger precision and a more favorable error correction asymmetry, and (iv) domain-specific fine-tuning yields substantial recall gains in client-heavy domains. These results support the viability of fully automated publishability determination in enterprise MT workflows.

LLM performance is highly sensitive to prompt design, yet whether automatic prompt optimization can replace expert prompt engineering in linguistic tasks remains unexplored. We present the first systematic comparison of hand-crafted zero-shot expert prompts, base DSPy signatures, and GEPA-optimized DSPy signatures across translation, terminology insertion, and language quality assessment, evaluating five model configurations. Results are task-dependent. In terminology insertion, optimized and manual prompts produce mostly statistically indistinguishable quality. In translation, each approach wins on different models. In LQA, expert prompts achieve stronger error detection while optimization improves characterization. Across all tasks, GEPA elevates minimal DSPy signatures, and the majority of expert–optimized comparisons show no statistically significant difference. We note that the comparison is asymmetric: GEPA optimization searches programmatically over gold-standard splits, whereas expert prompts require in principle no labeled data, relying instead on domain expertise and iterative refinement.

Current state-of-the-art Quality Estimation (QE) in machine translation relies on massive, proprietary LLMs, raising data privacy concerns. We demonstrate that smaller, open-source LLMs (<30B parameters) are a viable, cost-effective and privacy-preserving alternative. Using a single-pass prompting strategy, our models simultaneously generate quality scores, MQM error annotations, suggested error corrections, and full post-editions. Our analysis shows these models achieve highly competitive system-level correlations with human judgments that outperform traditional neural metrics, fine-tuned models, and human inter-annotator agreement, effectively approximating the capabilities of much larger proprietary LLMs.

Code-mixed and Romanized texts are widely used in digital content, yet they remain largely underexplored for many low-resource languages, including Sinhala. The scarcity of high-quality parallel data has limited progress on downstream tasks, such as machine translation and transliteration. We introduce SinMix2Mono, the largest manually annotated parallel training dataset, followed by the first gold standard benchmark and code-mixed transliteration ambiguity corpora for code-mixed romanized Sinhala to Sinhala conversion. The dataset comprises approximately 25,000 real-world sentences collected from social media, covering diverse domains and authentic code-mixing patterns. To ensure high-quality translations, we used an annotation pipeline that combined rule-based transliteration, LLM-assisted translation, and human validation. The golden test dataset, which includes 2549 sentences, and the code-mixed transliteration ambiguity test were validated by three annotators, yielding Gwet’s AC1 scores of 0.7465 and 0.7068, respectively. We benchmarked nine systems, including statistical, neural and commercial LLMs. SinMix2Mono provides a robust training and evaluation resource, establishing a strong benchmark for future research on Sinhala code-mixed translation and transliteration.

Sentence-level alignment systems have been developed and evaluated primarily on parallel data, leaving their behaviour across the broader parallel-comparable spectrum of real web content poorly understood. We present a stratified empirical study of alignment quality for Catalan-English using 300 document pairs across three parallelism bands defined by mean-max LaBSE cosine similarity. We compare four systems: a hierarchical alignment pipeline (DocAlign), an ablation with paragraph pre-filtering disabled (DocAlign-NoFilter), the flat aligner Vecalign, and a flat LaBSE greedy baseline. Evaluation uses human-annotated sentence pairs and coverage-weighted quality. Quality degrades at different rates by system type: hierarchical systems maintain usable-pair rates ranging from 25% to 51% on comparable data while flat systems collapse to 2-7%. Paragraph pre-filtering reduces output volume on comparable data while raising pair quality relative to the unfiltered ablation. Vecalign is statistically indistinguishable from the greedy baseline at all parallelism levels, suggesting that LaBSE embedding discrimination is the binding constraint on flat alignment quality. Failure mode analysis of 550 low-rated pairs identifies topical mismatch as the dominant failure mode, with structural noise concentrated in flat systems.

We present ForMaT (Format-Preserving Multilingual Translation), a parallel corpus of 3,956 PDFs across 15 language pairs that preserves original layout metadata proposed for multimodal machine translation. To ensure structural diversity in the dataset, we employ K-Medoids sampling over 45 geometric features, capturing complex elements like nested tables and formulas to focus only on visually diverse PDF documents. Our evaluation reveals that current MT systems struggle with spatial grounding and geometric synchronization, often losing the link between text and its visual context. ForMaT provides a benchmark for developing layout-aware translation models that integrate visual and textual context for high-fidelity document reconstruction.

Most machine translation datasets come with a certain level of noise, and strategies for handling such data need to be robust and efficient. Data selection and filtering are challenging and may depend on expensive language-specific tools that are not necessarily available, especially for low-resource languages. This paper looks at training strategies that combine cheap heuristic filters with curriculum learning to implement iterative procedures that robustly operate on raw noisy data without expensive prior preprocessing and data selection. The intuition is that we can cluster data into buckets with varying noise levels and use different sets of buckets at different stages of MT model training. We test various strategies and compare them to pre-filtering approaches for a diverse set of low-resource languages and conclude that curriculum learning can improve robustness but does not necessarily lead to improved translation performance. Overall, the experiments demonstrate the importance of proper experimental workflows, which cannot easily generalize from one language pair and scenario to another.

This study investigates the presence and mitigation of gender bias in English-Ukrainian machine translation (MT) models. We focused on the transfer of gender bias in two English-Ukrainian MT models, using sentences that contained professional occupation names. We evaluated two gender bias mitigation methods: 1) gender tagging of source sentences, and 2) gender bias correction by Lapa LLM, a Ukrainian large language model (LLM), using a dataset that we curated for our evaluation. Our results showed that both zero-shot models contained English-Ukrainian gender bias transfer, particularly for gender-stereotypical occupations. The gender tagging mitigation method demonstrated changes to the gender assignment in our data; however, these changes led to mixed results in gender bias correction. The Lapa LLM-correction method had promising results in demonstrating a considerable mitigation of bias in our evaluation set. Overall, our study contributes a framework for the evaluation of gender bias in English-Ukrainian translation that could potentially be applied to translation pairs in other languages.

Discourse-level translation remains a major challenge for machine translation (MT) systems, particularly for translation from Indian languages to English. This difficulty arises due to factors such as rich morphology, diverse syntactic structures, unmarked gender distinctions in pronouns, and the limited availability of discourse-aware training data. Existing evaluation benchmarks primarily focus on sentence-level translation quality and fail to capture important discourse phenomena such as pronoun resolution and lexical cohesion. To address this gap, we introduce IndicDISCO-MT, a parallel benchmark dataset covering translations from eight Indian languages such as Bengali, Gujarati, Hindi, Marathi, Kannada, Tamil, Telugu and Urdu to English. On top of this dataset, we are first to introduce DiscoAlign, a human annotated word to word alignment benchmark dataset, that captures correspondences between source and target words across languages. In addition, we propose two evaluation benchmarks, ProAlign and LexiAlign, designed to specifically assess the ability of large language models (LLMs) and MT systems to handle personal pronouns and lexical cohesion. Our evaluation of recent LLMs and MT systems on these benchmarks shows that although models achieve high overall translation quality, they still struggle to accurately preserve discourse-level phenomena. The proposed benchmarks provide a systematic framework for evaluating discourse aware translation and can facilitate the development of MT systems that generate more coherent and contextually consistent translations.

As machine translation increasingly requires deeper contextual, linguistic, and cultural understanding, multi-agent collaboration has emerged as a promising approach. Multi-agent debate (MAD) frameworks, in which multiple agents deliberate to produce a final output, have shown strong performance on objective tasks, but remain underexplored in translation, where multiple valid renderings often exist. We adapt three MAD frameworks for English-Japanese translation and evaluate them against strong generative baselines, reasoning-capable LLMs, and a prompt-based self-reflection baseline. Across general-domain and culturally grounded datasets, the Society of Mind (SoM) variant yields the strongest results in the English-to-Japanese direction, showing that zero-shot translations leave substantial room for improvement through structured deliberation. Yet the gains of debate are front-loaded: later rounds do not reliably improve quality and often reintroduce translation errors. Diagnostic and error-span analyses show that hand-designed debate protocols tend to over-revise already strong translations, leading to semantic drift and process-induced degradation. These findings highlight both the promise and the limitations of agentic translation, and suggest that effective debate-based systems require mechanisms for preserving strong intermediate outputs.

Literary translation is a difficult task that not only requires semantic accuracy but also stylistic richness and lexical diversity. Pretrained and supervised fine-tuned Large Language Models (LLMs) can over-rely on safe vocabulary choices, leading to translations that lack lexical variety. To address this problem, we propose a novel diversity-aware multi-objective Group Relative Policy Optimization (GRPO) framework that pushes the limits of open-source translation quality while increasing lexical diversity. We introduce two diversity-aware reward mechanisms, a Leave-One-Out (LOO) marginal contribution reward and a Self-BLEU penalty, balanced alongside neural quality metrics (COMET), lexical overlap (BLEU), and structural constraints. Through experiments on Turkish-English and German-English using Qwen3-14B, we show that our diversity-aware reinforcement learning approach successfully enhances lexical richness alongside translation quality. Our models achieve state-of-the-art open-source performance in literary translation, bridging the gap with leading commercial systems and demonstrating that policy optimization can effectively steer LLMs toward high-quality, lexically diverse outputs.

Machine translation (MT) systems continue to produce gender-biased translations. In a time where self-expression is paramount, mistranslations based on default behaviour and stereotyping can lead to harm for users of these systems. To better understand how these systems translate gender in the absence of clear gender cues, we need benchmarking resources that reflect gender ambiguous scenarios in a natural way. To this end, we present GAND, a gender ambiguous natural data benchmarking resource for MT consisting of English source sentences, specifically designed to analyse the influence of contextual cues on gender in translation. We leverage GAND to conduct an interpretability analysis: we translate a subset of GAND into two grammatical gender languages and extend these with manually crafted contrastive translations. A following feature attribution analysis reveals source words in context that inform the gender translation of an ambiguous referent entity in the target translation. As a newly introduced resource, GAND is designed to be benchmarked across diverse target languages and evaluated with a wide range of MT systems.

Recent advances in neural machine translation (MT) have spurred increased interest in evaluating translations beyond the sentence level, making it possible to assess discourse-level phenomena related to coherence and consistency. While existing metrics can be applied to multi-sentence spans, it remains unclear whether their scores truly capture document-level quality. We introduce MetaDocEval, an automatic contrastive test set for evaluating MT metrics across three language pairs (en–fr, en–es, en–de) when applied at the document-level. It targets a range of discourse-level phenomena and potential problems linked to translation at the document level. To evaluate how metrics behave as a function of context size, we apply them under a sliding-window protocol, varying the input from single sentences up to full documents. Our experiments show that no current metric genuinely captures document-level coherence: reference-based metrics overfit lexical overlap, reference+source metrics gain little from added context, reference-free encoders show brief context sensitivity before degrading on longer spans, and LLM-based scorers collapse beyond short inputs. A key finding is that reference access can be actively harmful for detecting discourse-level errors. Using short windows (≈ 3 sentences) offers the best trade-off between discourse error detection and score dilution.

EU legislation is equally authentic and legally binding in all 24 official languages, rendering high-quality translation a legal obligation rather than a mere choice. Therefore, high-quality language technology supporting translation processes in all EU languages are essential for language professionals at the European Parliament (EP). This paper investigates whether domain-specific fine-tuning of an open-weight Large Language Model (LLM) yields consistently larger quality gains on legislative text compared to generic text, in all 23 EU target languages from English. We evaluate ten experimental conditions: base model, in-domain and cross-domain fine-tuning, sequential generic-then-legislative fine-tuning, and zero-shot Claude Sonnet 4.6 as a proprietary reference. We analyse BLEU, chrF, TER, and COMET metrics on nearly 700,000 segments. Results confirm the hypothesis for all 23 languages: legislative fine-tuning enhances BLEU by +12.30 compared to +7.10 for generic fine-tuning, demonstrating a consistent advantage of +5.20 BLEU in all the metrics. The fine-tuned EuroLLM-22B decisively outperforms Claude Sonnet 4.6, Anthropic’s latest frontier model, on both domains, highlighting that targeted adaptation of a smaller open-weight model can surpass a state-of-the-art proprietary system. Cross-domain transfer within institutional domain is positive for all languages, with no catastrophic forgetting. Low-resource languages such as Irish and Maltese benefit the most from fine-tuning, while a divergence between BLEU and COMET rankings for some languages underlines the need of evaluation metrics alongside traditional measures.

As state-of-the-art machine translation models saturate standard benchmarks, the field needs more challenging evaluations to distinguish between models of varying quality. We propose augmenting existing benchmarks to increase translation difficulty by combining adversarial optimization with a differentiable translation difficulty estimator. Our Adversarial Translation Optimization (ATO) uses gradients from a combined difficulty and fluency objective to iteratively replace tokens. Because each step branches over candidate substitutions at every position, optimization becomes a tree search problem, which we address with Beam Search. ATO offers a gradient-based alternative to LLM-based dataset creation without LLM prompting, expensive human curation, or task-specific model training. Our ATO-modified benchmark lowers average translation quality (xCOMET) from 0.93 to 0.82, compared to 0.88 for paraphrasing and 0.86 for a zero-shot baseline. Through human evaluation we confirm that the modified texts remain reasonably natural while being substantially harder to translate. We release two datasets of 200 English texts each, generated by our methods, as well as the code.

Human evaluation of MT is essential but exhibits substantial annotator variability that limits evaluation reliability and super- vised learning. Rather than treating dis- agreement as noise or correcting it through protocol changes, we analyze its structure via learned severity classifiers. Across training regimes defined by base- line model reproducibility, we observe in- ternally coherent but mutually incompati- ble severity mappings: models trained on one regime produce confident predictions within that regime but reduced separability on the other. Margin–correctness analysis shows that instability is not uniformly low confidence; separability depends on align- ment between model-internalized and hu- man annotation regimes. These results indicate that unstable MT evaluation regions arise primarily from competing severity interpretations rather than intrinsic example difficulty. Model– annotator disagreement therefore provides a practical signal for identifying unstable evaluation regions during MT evaluation.

Term injection in machine translation is undergoing a paradigm shift in the era of large language models (LLMs). Although recent shared-task results suggest near-saturation for sentence-level term injection from pre-defined glossaries, it remains unclear whether this also holds in more challenging settings. We address this question with a custom test set for legal translation from Italian into South Tyrolean German, a non-dominant and under-resourced language variety. We cover three terminology challenges: simple term injection, localisation of abbreviated forms, and homonym disambiguation. We focus on Reasoning Models (RMs) leveraging Test-Time Scaling, comparing them with different architectures and contributing a human analysis of reasoning traces. We find that reasoning offers little benefit for simple term insertion, but yields clear gains for semantically complex cases such as homonym disambiguation. However, human evaluation of reasoning traces shows that these gains do not necessarily reflect robust and factually grounded translation-specific reasoning. We further show that without external terminological resources, even state-of-the-art RMs struggle to retrieve correct terminology for a non-dominant variety, while NMT small models remain competitive when trained on in-domain bilingual corpora. Based on these findings, we propose data collection strategies for inducing translation-specific reasoning, frameworks for adapting to and evaluating terminology across many language varieties, and terminology challenges beyond simple term injection.

Preserving affective nuance remains a challenge in Machine Translation (MT), where semantic equivalence often takes precedence over emotional fidelity. This paper evaluates the performance of three state-of-the-art Small Language Models (SLMs) – EuroLLM, Aya Expanse, and Gemma – in maintaining fine-grained emotions during backtranslation. Using the GoEmotions dataset, which comprises Reddit comments across 28 distinct categories, we assess emotional preservation across five European languages: German, French, Spanish, Italian, and Polish. Specifically, we investigate (i) the inherent capability of these SLMs to retain emotional sentiment, (ii) the efficacy of emotion-aware prompting in improving preservation, and (iii) the performance of ModernBERT as a contemporary alternative to BERT for emotion classification in MT evaluation.

Adapting large language models to specialized domains remains challenging due to the computational cost of full finetuning and the limited availability of domain-specific parallel data. We present a systematic framework for parameter-efficient domain adaptation using Low-Rank Adaptation (LoRA) geared towards efficient learning in low-resource scenarios. Our method combines data-scaling analysis, dual-track hyperparameter optimization, and competitive benchmarking. We evaluate our approach on the low-resource English–Norwegian petroleum translation domain using a distilled version of NLLB and parallel data from the Norwegian Petroleum Directorate. Our adapted model achieves 61.48 BLEU (+24.62 over the base model) and 0.9298 COMET, while updating <0.4% of parameters. Our results provide a reproducible and computationally efficient blueprint for domain adaptation in neural machine translation, particularly for specialized and resource-constrained domains.

Knowledge distillation (KD) compresses large teacher models into smaller student models by transferring soft labels or intermediate activations. While effective in general domains, KD alone falls short in specialised machine translation (MT) settings, such as biomedical translation. The student inherits only the teacher’s compressed knowledge and lacks access to external domain information. Moreover, standard KD typically relies on abundant parallel data, which is often unavailable in domain-specific scenarios. To address these limitations, we combine KD with retrieval-augmented generation (RAG) in a few-shot setting. We propose a retrieval-augmented enhanced few-shot KD framework for French-to-English biomedical translation task. The student learns to retrieve relevant in-domain knowledge from an external database, complementing the teacher’s supervision. We design and compare several retrieval strategies to enhance student capacity. Experiments show that with our terminology-aware retrieval-based methods, the student achieves performance comparable to or better than the teacher, while preserving translation quality and efficiency.

Conventional retrieval-augmented neural machine translation (RANMT) systems leverage bilingual corpora, e.g., translation memories (TMs). Yet, in many settings, monolingual corpora in the target language are often available. This work explores ways to take advantage of such resources by directly retrieving relevant target language segments, based on a source-side query. For this, we design improved cross-lingual retrieval systems, trained with both sentence level and word-level matching objectives. In our experiments with two RANMT architectures, we assess of such cross-lingual objectives in a controlled setting, reaching performances that match those of standard TM-based models. We also showcase our method on real-world settings, using much larger monolingual corpora, and observe strong improvements over both the baseline setting, and general-purpose cross-lingual retrievers.

We present an evaluation of several state-of-the-art machine translation systems supporting terminology constraints in the English–Finnish translation direction. We first perform a meta-evaluation, in which we critically evaluate the evaluation metrics we use, including the questions asked from human evaluators and the automatic evaluation methods. We find that common metrics such as term accuracy and TERm do not agree with the human evaluators’ judgement on the correctness of the terms, while LLM-as-a-judge shows promise even though it does not agree with the human evaluators on all questions. We then compare the evaluated systems based on the human evaluation results, LLM-as-a-judge, COMET, and chrF2. We find that of the systems considered, soft constraint methods, including a term-trained model and an LLM, perform better than hard constraints forced using a constrained beam search.

Evaluating the translation of multi-word expressions (MWEs) remains a major challenge for Machine Translation (MT), particularly in audiovisual subtitling, where idiomatic meaning and cultural context are essential for adequacy. This study investigates both the ability of state-of-the-art MT systems to translate Spanish MWEs into English and the extent to which current automatic evaluation methods reflect expert human judgment. We introduce ALMO-MWE, a dataset of 235 MWEs extracted from four films by Pedro Almodóvar to evaluate four MT systems using automatic metrics, LLM-as-a-judge approaches, and professional human assessment. Our results reveal a substantial mismatch between traditional automatic metrics and human judgments: n-gram-based metrics show near-zero correlation with expert evaluation and only limited discriminative capacity. In contrast, neural metrics and LLM-based judges exhibit substantially stronger alignment with human assessments, with GPT-OSS achieving the highest overall correlation. These findings highlight fundamental limitations of surface-form metrics for culturally and contextually sensitive translation phenomena and underscore the need for context-aware evaluation frameworks when assessing the translation quality of MWEs in audiovisual translation.

User-generated content (UGC) is characterised by frequent use of non-standard language, from spelling errors to expressive choices such as slang, character repetitions, and emojis. This makes evaluating UGC translation challenging: what counts as a "good" translation depends on the desired standardness level of the output. To explore this, we examine the human translation guidelines of four UGC datasets, and derive a taxonomy of twelve non-standard phenomena and five translation actions (NORMALISE, COPY, TRANSFER, OMIT, CENSOR). Our analysis reveals notable differences in how UGC is treated, resulting in a spectrum of standardness in reference translations. We show that translation scores of large language models are highly sensitive to prompts with explicit UGC translation instructions, and that they improve when they align with the dataset guidelines. We argue that fair evaluation requires both models and metrics to be aware of translation guidelines. Finally, we call for clear guidelines during dataset creation and for the development of controllable, guideline-aware evaluation frameworks for UGC translation.

Automatic Post-Editing (APE) is a widely studied approach for enhancing the output quality of Neural Machine Translation (NMT) systems. While most prior work has focused on general-purpose APE, the potential of domain-specific APE, such as for personalized or specialized content, remains underexplored due to the scarcity of domain-labeled training data. In this work, we investigate domain adaptation for APE using adapter-based methods. Our proposed multitask learning-based domain adaptation framework includes the use of a domain classifier to get a weighted combination of parallel domain-specific adapters at inference time, without requiring prior domain knowledge. This design allows the model to leverage cross-domain similarities, making it especially robust in low-resource domain scenarios. Our experimental results on English–German, English–Marathi, and English–Tamil pairs across different domains for each pair show substantial improvements over their respective general-purpose APE baselines. To facilitate further research, we will release human-annotated domain labels for triplets in WMT22 English–Marathi, and WMT24 English–Tamil APE datasets and the code.

We present two open-weight translation models for Estonian and its low-resource "relatives" in the Finno-Ugric language family. The training data includes 12 languages paired with Estonian as well as 23 more Finno-Ugric languages and varieties, ranging from mid-resource examples with tens of thousands of speakers to extremely low-resource critically endangered languages with less than a hundred speakers. The translation models use Unbabel Tower+ 2B and 9B as their starting point. We compare their performance on two benchmarks to DeepL and GPT-5.2 and show that in most cases we surpass the quality of DeepL and match or nearly match the quality of GPT-5.2’s output with just a fraction of the parameters. Among other contributions we also restore the paragraph structure of a massive synthetic multiparallel corpus for Estonian translation and use it in training the models. The resulting models, training scripts and training data are released openly (to be made public upon de-anonymization).

E-commerce product titles often include redundant information that negatively impacts the user experience. Removing repeated words through restructuring and paraphrasing can make titles more concise and improve readability. While large language models can optimize titles, their computational cost makes them impractical for large-scale applications. In this paper, we first analyze the sources of repetition in multilingual product titles, then present LocRegen, a system that uses smaller language models to efficiently remove redundancies while preserving essential product attributes. Our experiments across five languages show that LocRegen with a 7B model substantially outperforms a 47B mixture-of-experts model: LocRegen achieves a 2.4% redundant title rate compared to 3.5% for the 47B model, and maintains a 3.8% overall error rate across all error categories including key product attribute omission compared to 8.4% for the 47B model. These results demonstrate that LocRegen delivers superior performance on cost-effective hardware with acceptable latency, making it practical for large-scale deployment where much larger models would be computationally prohibitive.

In-context learning is a method for improving machine translation in Large Language Models, but its performance is sensitive to the quality of the few-shot example selection. Current retrieval strategies use semantic similarity by computing sentence embeddings, and these methods often require significant computational overhead and specialised expertise. We evaluate the impact of retrieval strategies on translation performance in a specialised domain, comparing traditional, token-based fuzzy matching against semantic sentence embeddings. We use a medical corpus from the European Medicines Agency (EMEA) for the English-Romanian and English-German language pairs, and we evaluate translation quality with automatic metrics and manual evaluation. Our results show that 1-shot and 5-shot prompting significantly outperforms the 0-shot baselines for quality in automatic evaluations for both language pairs, and in manual evaluation for English-German. For the English-Romanian pair, the average scores of the manual evaluation for both quality and ranking follow the same trend, but with no statistical significance. Token-based fuzzy matching overwhelmingly has higher automatic quality scores than embedding-based retrieval.

This study investigates whether professional translators can reliably identify short stories generated in Italian by artificial intelligence (AI) without prior specialized training. Sixty-nine translators took part in an in-person experiment, where they assessed three anonymized short stories - two written by ChatGPT-4o and one by a human author. For each story, participants rated the likelihood of AI authorship and provided justifications for their choices. While average results were inconclusive, a statistically significant subset (16.2%) successfully distinguished the synthetic texts from the human text, suggesting that their judgements were informed by analytical skill rather than chance. However, a nearly equal number misclassified the texts in the opposite direction, often relying on subjective impressions rather than objective markers, possibly reflecting a reader preference for AI-generated texts. Low burstiness and narrative contradiction emerged as the most reliable indicators of synthetic authorship, with unexpected calques, semantic loans and syntactic transfer from English also reported. In contrast, features such as grammatical accuracy and emotional tone frequently led to misclassification. These findings raise questions about the role and scope of synthetic-text editing in professional contexts.

Translators at the Canadian Parliament currently have access to a neural machine translation system as an optional tool integrated into their translation environment, whose output they can use for postediting (rather than translating from scratch); this provides a valuable opportunity to study the dynamics of MT adoption in professional settings. We report on a user study that investigates how and why translators choose to interact with this tool. Using a mixed-methods approach, we examined both human and technical factors that influence the adoption or non-adoption of the system. Drawing on our findings, we advocate for a user-centred approach to MT integration within professional translation workflows.

Building on our previous work, this paper develops practical, low-barrier methods for freelance translators and smaller language service providers to evaluate translation technologies using rigorous yet accessible analytic methods. Here we address a high-stakes, specialized need: offline translation for confidentiality-sensitive domains in which privacy constraints preclude the use of cloud-based engines and commercial LLMs. We expand the Reeve Foundation Trilingual Corpus (RFTC) used in our previous work into a multilingual corpus (RFMC) by adding sentence-aligned German and Simplified Chinese reference translations. We then benchmark several locally runnable language models (via Ollama) across four language directions on 1000+ sentences selected from this corpus. We use consistent single-prompt calls without fine-tuning or domain adaptation, comparing local LLM outputs against commercial NMTs (DeepL, Baidu), a frontier LLM (GPT-5.2), and professional-grade local NMT systems (OPUS-CAT, NeuralDesktop, Promt). Automatic evaluation is conducted with MATEO. Results reveal substantial variation in local LLM performance across language directions and model sizes. The best local LLMs match or surpass local NMT systems and a frontier LLM, though they remain behind top commercial NMTs. These findings underscore the viability of carefully selected local LLM translation for privacy-constrained professionals and inform future research on model scaling and multilingual capability.

Machine Translation (MT) has become a default language access tool in public and third sector organisations serving multilingual communities. However, how organisations, and their staff, actually use it, and their opinions about it, remain largely undocumented. This paper reports findings from charities, NGOs, community organisations and local government authorities in the West Midlands (UK), one of the most linguistically diverse areas of the country. The results indicate that MT use is widespread, informal, and driven by necessity rather than informed decisions or policies. Google Translate is the preferred tool; policies about MT use are rare, and confidence in translation quality is limited. Risk perception varies across the sector: local government respondents identify the widest range of concerns, including legal and medical, while third-sector organisations suggest a pragmatic approach. However, greater risk-awareness does not lead to greater governance, pointing to a gap between individual MT literacy and institutional accountability. Based on this, we propose some recommendations for how organisation serving multilingual communities should approach MT implementation and training.

This paper reports on a mixed-methods pilot study examining the potential of a large language model (LLM) for translating tourism promotional texts (TPTs), in comparison with a conventional neural machine translation (NMT) system, from English into Arabic. Four professional translators participated in a post-editing experiment followed by semi-structured interviews involving cue-based retrospection. The post-editing task aimed to provide empirical evidence of the effort involved in working with TPTs, while the interviews sought to capture participants’ judgments and subjective evaluations of the outputs. Overall, most participants exerted less effort post-editing LLM-generated outputs to a publishable standard compared to NMT outputs. They perceived the LLM outputs to be more creative, with creativity manifested through non-literal translations and aesthetic augmentation, while also noting that the outputs were unpredictable and far from perfect; in contrast, the NMT outputs were generally viewed as more informative yet lacking the promotional appeal needed for TPTs. The paper concludes by outlining the implications of the findings and suggesting several avenues for future research.

This study examines errors in a ChatGPT-mediated translation of a German economic text on inflation into Spanish, post-edited by 20 translation students. The analysis classifies 132 annotated instances by error origin (ChatGPT-generated versus student-introduced during post-editing) and by lin-guistic category. Results show that termi-nology is the highest-risk domain across the entire workflow (34.1%), followed by tense/aspect (15.2%) and style (13.6%). ChatGPT-related errors account for 50.8% of all instances, while student-introduced errors through over-editing represent 21.2%. A further 28.0% reflect acceptable alterna-tive reformulations. Students tend to trust fluent machine output even when it con-tains subtle semantic distortions, yet they also over-edit segments that are already ac-ceptable. The findings highlight three di-dactic priorities: developing LLM-based MT literacy, strengthening decision-making strategies in post-editing, and fostering gen-re- and domain-sensitive editing compe-tence. Implications for translator training and structured post-editing protocols are discussed.

As MT quality increases, interest in enhanced post-editing features such as QE-derived error highlights is growing, yet evidence for their usefulness remains limited. In this work, we explore the usefulness of LLM-derived error highlights and correction suggestions based on automatic post-editing (APE). We conduct a study where professional translators (En-Nl) post-edit translations using APE error highlights and correction suggestions. While no condition yielded productivity or quality gains compared to regular PE, APE highlights were better received than QE-derived highlights, and correction suggestions improved overall user experience.

This paper is the one of the first in-depth accounts of the use of machine translation (MT) by journalists. It reports on a study of Finnish journalists that was conducted mostly in 2024 and comprised an online survey with 68 responses and interviews with 10 journalists. Results revealed that participants fluently integrated MT into a variety of journalistic processes, with an emphasis on using it for assimilation and dissemination; that they relied largely on traditional online MT tools and tended to employ MT mostly with languages they have some competence in; and that they had some awareness of risk and strategies for mitigating it, but could benefit from guidelines and training on working with MT. The article contributes to research on the users and contexts for MT and, specifically, to its paraprofessional use in workplaces.

This article investigates the performance of automatic evaluation metrics (AEMs) and LLM-as-a-judge evaluation on literary translation across multiple languages, genres, and translation modalities. The aim is to assess how well these tools align with professionals when evaluating translation, creativity (creative shifts & errors), and see if they can substitute laborious manual annotations. A dataset of literary translations across three modalities (human translation, machine translation, and post-editing), three genres and three language pairs was created and annotated in detail for creativity by experienced professional literary translators. The results show that both AEMs and LLM-as-a-judge evaluations correlate poorly with professional evaluations on creativity, with LLM-as-a-judge showing a systematic bias in favour of machine-translated texts and penalising creative and culturally appropriate solutions. Moreover, performance is consistently worse for more literary genres such as poetry. This highlights fundamental limitations of current automatic evaluation tools for literary translation and the need to create new tools that do not frequently consider out of routine translations as errors.

This study examines the machine translation of audio descriptions (AD) as an alternative to producing new AD for audiovisual formats in a foreign language. To assess acceptance and comprehensibility among German users, a survey was conducted with blind and visually impaired participants, examining key AD strategies, such as character description and naming, facial expressions and gestures, and spatio-temporal settings. Participants compared machine-translated English AD with original German AD and provided feedback on these aspects. Results showed overall acceptance of the translated AD, although the original was generally preferred. Findings suggest that AD translation is feasible for the German audience, but further studies are needed on machine translation, production costs, as well as larger-scale user studies.

This paper presents findings from a survey on the use of automatic translation in Finnish public services, conducted in autumn 2025. Adapted from a similar survey conducted at the University of Bristol, the present survey focused on users whose main professional activity is not translation or interpreting. The study analyzed those professionals’ habits around automatic translation use, the purposes and the contexts involved, and the respondents’ satisfaction and confidence in using automatic translation. Approximately half of the respondents reported using automatic translation at least once a week and across a range of scenarios, including public-facing situations. While the survey sample is not representative of the Finnish public sector as a whole, the data suggests that automatic translation may play a role in the everyday work of at least some public service employees.

This work benchmarks safety across several large language models (LLMs) and compares their performances through red teaming, which simulates adversarial attacks and identifies vulnerabilities in the systems. Using two public datasets and a proprietary dataset, the models were tested with three purposes. First, a red teaming test was conducted to establish a safety comparison between five models in English and Portuguese. The results revealed that, in general, Sugarloaf 3.1 is the safest model, but that Vesuvius 4.0 slightly outperforms it in Portuguese, also revealing that both outperform GPT-4o. Afterwards, three models were tested with one guardrailing prompt, that encourages safe interactions, and two content moderation prompts, in both languages, to understand the strengths of the current guardrails, as well as the effectiveness of the content moderation task. The results show that current guardrails are sufficient, notwithstanding room for improvement (particularly for Portuguese), but that the performance of the content moderation task was substandard, even for the best performing model – GPT-4o. Finally, the 3.0 TowerLLM models were tested in English to evaluate the effect that tokens and temperature have on the output, revealing that an intermediate token limit leads to safer responses while a higher temperature causes performance degradation.

This paper investigates how post-editors of literary texts react and respond to the way metaphors have been translated by Neural Machine Translation (NMT) and Large Language Models (LLM). The results show that one in three metaphors in the output were changed by the posteditors, demonstrating that the translation of figurative language is indeed problematic in literary MT (LitMT). The responses indicate that the post-editors were aware of overly literal translations, though mostly for multiword expressions. Moreover, at times they found it difficult to determine whether solutions were acceptable. They rated the overall quality of the MT output as quite poor and stated that the post-editing was more work and more effort than it would have been translating from scratch. This supports previous studies arguing that post-editing constrains translators in their creativity and diminishes their sense of text ownership.

AI language technologies (AILTs), increasingly enabled by large language models (LLMs), are becoming embedded in multilingual healthcare workflows for translation, rewriting, documentation, interpreting, and messaging in language-discordant settings. Yet fluent output is not the same as clinically safe or equitable communication: performance varies across languages, accents, tasks, and workflows, and efficiency gains can hide errors, reduce traceability, and shift responsibility across clinicians, translators, interpreters, and health systems. This narrative review synthesises recent peer-reviewed evidence across written communication, spoken communication, and emerging agentic workflows. Using the Human-Centered AI Language Technology (HCAILT) lens, it examines capabilities, evaluation practices, implementation patterns, and recurrent errors through reliability, safety culture, and trustworthiness. We identify key convergences and contradictions in the literature and propose seven grand challenges for the next phase of research and deployment. Progress, we argue, requires not only better models but also accountable sociotechnical design, calibrated human oversight, and stronger collaboration across MT/NLP, translation studies, HCI, clinical practice, implementation science, and policy.

Cultural heritage institutions increasingly disseminate research and interpretive materials globally, but multilingual dissemination is constrained by limited budgets and staffing. In terminology-dense domains such as rock art, translation quality depends on accurate, consistent specialised terms, and small lexical errors can mislead non-specialists and reduce reuse. We compare three English MT setups for a Spanish academic rock art text: (1) DeepL as a strong NMT baseline, (2) Gemini-Simple (LLM with a basic prompt), and (3) Gemini-RAG (the same LLM with glossary-augmented prompting via lightweight retrieval of term pairs). Using PEARMUT, we conduct a human evaluation via (i) multi-way Direct Assessment (0–100) on 91 segments (1,743 Spanish words) and (ii) targeted terminology auditing with a restricted MQM taxonomy. Gemini-RAG yields the highest exact-match terminology accuracy (81.4%), versus Gemini-Simple (69.1%) and DeepL (64.4%), while preserving overall quality (mean DA 85.3 Gemini-RAG vs. 85.2 Gemini-Simple) and outperforming DeepL (80.3). These results show that glossary-augmented prompting is a low-overhead way to improve terminology control in cultural-heritage translation, provided that institutions maintain minimal terminology resources and lightweight evaluation procedures.

The aim of this paper is to propose a framework to support literary translators—particularly poetry translators—in making effective use of large language models (LLMs) through established prompt engineering strategies applied to both pre-translation and translation stages. The paper illustrates these strategies using poems characterized by multiple layers of syntactic, semantic, phonological, and cultural complexity, and discusses how LLMs perform in response to each prompting technique. It also engages with the longstanding claim that poetry translation is a purely human endeavour and cannot be computer-assisted, arguing instead that LLMs, rather than replacing human creativity, have the potential to extend it.

The integration of large language models (LLMs) into translation practice has substantially reshaped translation workflows (Kornacki and Pietrzak, 2025). Since translation quality depends partly on how these models are prompted, prompt design deserves closer attention as a key stage of the LLM-augmented translation process. This study investigates Spanish<>Italian translation with GPT 5.1 in the advertising and biomedical domains. It examines whether prompt language affects the quality of translations generated with the GPT 5.1 model, and specifically whether prompts written in the target language outperform prompts written in English, the language most prevalent in the model’s training data (Armengol-Estapé et al., 2022). Three prompt templates, varying in complexity and informational content, were tested. The translations were first screened for textual similarity, and only the translations generated from the template that produced the greatest variation across outputs were subsequently selected for human evaluation. Human judgments were collected through a pairwise-comparison task. The findings indicate that prompts written in the target language tend to yield higher-quality translations compared to prompts written in English.

This paper examines user reactions to the launch of the machine translation (MT) feature on Xiaohongshu, a Chinese social media and e-commerce platform, in January 2025. Drawing on a dataset of 6,723 comments collected from 11 official posts promoting the translation function, this paper combines sentiment analysis with thematic analysis to investigate how users perceived and experimented with this function. Results show that reactions were generally positive, although concerns about functionality, accessibility, and translation accuracy were also expressed. In addition, users actively tested the function with inputs that fail to represent everyday online communication, including stand-alone words and phrases, abbreviations, internet slang, and symbolic or encoded forms. Successful decoding of these texts elicited positive responses, while testing of more conventional language remained fairly limited. This could lead to uncritical acceptance of MT outputs by users, highlighting the importance of closer collaboration among computer scientists, translation scholars, and platform designers to improve MT performance and promote informed user engagement in real-world deployment of MT functions.

The present paper focuses on the analysis of automatic subtitles produced with three different systems. We compare the outputs among each other paying attention to the categories of quality derived from audio-visual translation quality research. Besides that, we also consider comprehensibility of the produced subtitles. Additionally, we analyse the automatic evaluation scores to assess the overall quality. Our results show that automatically generated subtitles subtitles remain below human standards in quality and comprehensibility.

AI-based technologies have disrupted the translation profession. While surveys such as the annual European Language Industry Survey have provided regular reports on the situation in Europe, less is known about the effects of AI-based translation tools on professional translators working elsewhere. This study reports on a survey that was conducted with support from the professional translators association of Quebec in Canada (Ordre des traducteurs, terminologues et interprètes agréés du Québec). One hundred and seventy-five completed surveys, along with additional partial responses, were analyzed. This article reports on questions relating to two broad categories: general perceptions about AI’s influence on the translation profession, and the evolution and sustainability of the profession. Where relevant, the results of this survey are compared to those from other parts of the world. Findings show that while Quebec translators face many issues that are similar to those faced by translators in regions such as the United Kingdom, France, Belgium, Switzerland, and Europe more generally, there are subtle differences also, such as the tendency of many Quebec translators to work as generalists, the comparatively low number of Quebec translators working in the Entertainment, Arts and Culture domains (which are showing signs of growth elsewhere), and the large number who seem hesitant to supervise interns moving forward.

Specialised translation relies on the use of documentary and terminological resources, including corpora. These resources are particularly useful for terminology. However, their compilation and exploitation have several limitations: they require time, technical skills and access to data that can be difficult to collect. This study examines the extent to which LLMs can assist specialised translators in finding equivalents from English to French. We evaluate four proprietary models, GPT-4o, GPT-5.2, Claude Sonnet 4.5 and DeepSeek, in two specialised domains, Earth, Environmental and Planetary Sciences (EEPS) and Natural Language Processing (NLP). The experiment is based on 80 terms per domain and compares two prompting strategies: a terminology and a translation mode. The results highlight clear differences between models, prompting strategies and, to a lesser extent, domains. Claude Sonnet 4.5 achieves the best results in the most favourable configuration, while DeepSeek stands out for its greater stability. Analysis of confidence estimates also shows that they are only a partial indicator of terminological accuracy. Overall, the findings suggest that LLMs can be useful tools for specialised translators, but cannot, at this stage, replace specialised corpora. This research therefore paves the way for future work on the real practical usefulness of LLMs for specialised translators in work and educational contexts.

This study explores the influence of two prompting strategies on the lexical and syntactic metrics of the LLM-powered machine translations (MTs) of a corpus of 18~British editorials into French as well as their impact on the edit types made by Master’s translation students post-editing from a representative editorial of the corpus, as evaluated using the MTPEAS taxonomy. Quantitatively, the prompt specifically requesting more syntactic and lexical variety leads to significantly higher syntactic and lexical metrics scores in the MTs, but differences remain significant only for lexical metrics in the post-edited versions of the representative editorial. Qualitatively, we show that students post-editing from an MT featuring more idiomatic rephrasings and fewer syntactic calques (as opposed to an MT that is structurally closer to the source text) seem to make fewer edits overall, leave more MT errors unaddressed, and make fewer successful edits.

This study examines the influence of prompt language and translation theory-driven prompt design on the quality of Spanish–Chinese editorial translations generated by GPT-5.2. A parallel corpus of four EL PAÍS editorials was translated under 48 experimental conditions (4 prompt types × 3 prompt languages × 4 articles). Translation quality was assessed using BLEU and BERTScore-F1 for automated evaluation, alongside human evaluation based on the Multidimensional Quality Metrics (MQM) framework. Automated metrics identified the baseline prompt (BASE) as the best-performing condition, whereas human evaluation ranked the brief-oriented prompt (BRIEF) highest (MQM: 8.66 vs. 7.84), a reversal attributed to the single-reference constraint inherent in automated measures. Subtype analysis indicated that translation theory-driven prompts selectively reduced Awkward style errors, whereas Unidiomatic style errors persisted consistently across conditions. Prompt language exhibited negligible impact under both evaluation paradigms. These results indicate that translation theory-driven prompts are advantageous for language learners seeking high-quality editorial translations and underscore the necessity of human evaluation for accurately assessing LLM translation quality in this domain.