This paper introduces PSMT-eval, a new dataset for machine translation (MT) evaluation in Punjabi Shahmukhi, a low-resource language spoken by over 80 million people. We constructed an English–Punjabi Shahmukhi parallel corpus through pivot-based human translation, where the Urdu side of existing English–Urdu datasets was manually translated into Punjabi Shahmukhi. Four widely used English–Urdu datasets were selected: tatoeba-test.eng-urd, FLORES101, IN22, and MKB, with total count of 5182 sentences. To assess the dataset, we evaluated translations generated by state-of-the-art large language models (LLMs), including GPT-5, GPT-OSS-120B, Claude 4 Sonnet, LLaMA-3.3 70B Instruct, and Qwen-3 32B. Results show that only Claude 4 Sonnet achieved competitive performance, with a maximum SacreBLEU score of 17.51 on IN22. GPT-5 followed with a maximum SacreBLEU score of 10, while other models performed poorly. Further analysis highlights major challenges in Punjabi Shahmukhi MT. To the best of our knowledge, this is the first benchmark dataset specifically for MT evaluation in Punjabi Shahmukhi. PSMT-eval is a valuable resource for advancing low-resource MT research and is publicly available to support future work.

Romanized Hindi has no standard spelling: the same word may be typed nahi, nahin, or nai, and no formal romanization rule or general-purpose transliteration system captures this typed convention well. Transliteration quality therefore depends as much on which spelling a user would actually type as on phonological plausibility. We present LIPI, a compact, encoder-only Devanagari-to-Roman transliteration model (~0.5M parameters) that represents each akshara compositionally from its ordered Unicode codepoints, expands it into six learned CTC emission slots, and encodes the frames with a two-layer bidirectional GRU under a connectionist-temporal-classification objective. The model is trained on a usage-ranked lexicon we build only from the training partitions of Dakshina, Xlit-Crowd, and Aksharantar, ranked by an attestation-gated social-usage prior so that a frequent Roman spelling is preferred only when it is also attested as a transliteration of the source word. LIPI is on par with the much larger multilingual IndicXlit baseline on general word transliteration (Aksharantar) and far stronger on naturally typed queries (FIRE 2013: 73.2 vs. 34.8 exact match), roughly doubling both baselines’ word match on Dakshina and IndoNLP sentences while remaining small enough for on-device use. The word-level gains are statistically significant and hold on words unseen in training, and an emission-slot ablation isolates the learned expansion’s contribution. The model, lexicon, and full benchmark results are publicly released at https://github.com/ABHISHEKgauti25/lipi.

In Japanese–English machine-translation,especially for creative and localized content, standard MT metrics typically summarize translation quality as scalar scores, but provide little diagnostic information about how style changes in translation. This is limiting because politeness, directness, interpersonal stance, and character voice are encoded differently across Japanese and English, and stylistic change may reflect localization-oriented adaptation rather than error. We propose the Style Diagnostic Profile (SDP) framework: SDP assigns each utterance a seven-dimensional bipolar style vector, and 𝛥sdp, the difference between target and source SDP vectors, represents translation-induced stylistic change as a multi-axis shift vector. To evaluate the framework, we first validate an LLM-based SDP scorer against bilingual human annotations on professional JA→EN subtitles and show that it approximates human style judgments. Using Style25, a controlled JA↔EN style-perturbation benchmark, we demonstrate that different perturbation categories occupy distinct regions in 𝛥sdp space. As a case study, we analyze professional subtitles and observe stylistic adaptation rather than simple style copying. These results position the SDP framework as a diagnostic layer complementary to adequacy-oriented MT evaluation.

Multilingual translation requires a model to preserve meaning while producing text in a requested language. We present HEXABEE-4B, a translator adapted from Gemma 3 4B IT through three stages: Wikipedia continued pretraining, joint multilingual answering and answer-text translation, and bidirectional translation specialization. The intermediate stage connects two tasks through shared multilingual answers: answering a question in the requested language and translating an answer into that language. It comprises 19.46 million answering records, including 5.76 million cross-language question–answer pairs, and 10.70 million answer-text translations. The final stage uses 658,274 forward and reverse synthetic translation examples. On 36,572 synthetic test examples spanning all 30 directions among Arabic, Chinese, English, Hebrew, Persian, and Spanish, HEXABEE-4B achieves direction-macro chrF++ of 73.75, compared with 57.11 for Gemma 3 4B IT and 66.83 for Gemma 3 12B IT. BLEU, TER, COMET, and a matched model-judged subset support the same overall ranking. These results demonstrate improved translation on the evaluated synthetic distribution, with chrF++ gains in every evaluated direction over both baselines.

Serial subtitle translation requires work-specific terminology and translation conventions to be maintained across episodes, despite scarce bilingual supervision. Previously finalized subtitles provide both adaptation data and translation memory, but our experiments show that the benefit of combining supervised adaptation and retrieval varies across series. We introduce SerialDistill, which combines LoRA-based domain adaptation with retrieval-conditioned on-policy distributional distillation (OPD). A Qwen3-8B student generates translation trajectories, while a frozen Qwen3-14B teacher provides token-level supervision under the same bilingual context retrieved exclusively from previously finalized episodes. On Japanese-to-Chinese translation across two anime series, SerialDistill improves over the non-distilled LoRA-8B+RAG baseline by up to 3.80 chrF and 4.15 BLEU, measured on pooled held-out episodes. The student also outperforms the larger retrieval-augmented teacher, which has not undergone domain adaptation. These results support retrieval-conditioned distillation as an additional adaptation stage for data-scarce serial subtitle translation.

Existing automatic machine translation evaluation metrics, such as COMETKiwi and MetricX, showed relatively low correlations with human evaluation in the WAT2025 Patent Claim Translation/Evaluation Tasks. We propose a rubric-based LLM-as-a-Judge method for patent claim translation evaluation. Based on a patent-specific rubric consisting of five evaluation dimensions with corresponding checklist items, we instruct a GPT model to assess translation quality. On a publicly available human evaluation dataset derived from the WAT2025 Patent Claims Translation/Evaluation Tasks test set, the proposed method achieves the highest Pearson and Spearman correlation coefficients with human evaluation among all evaluated methods.

Low-resource Asian machine translation draws on monolingual web text from Common Crawl. We ask what share of Bengali Common Crawl a detector flags as machine output, and whether it moves with translation or generation. We introduce BanglaGen-Bench, a 2,700-document Bengali benchmark with human-written, English-to-Bengali machine-translated and LLM-generated classes, and a stylometric detector that needs no Bengali-specific tooling. Translatedness did not transfer between the two NMT systems we tested: a detector trained only on NLLB-200 output recognises mBART-50 output 5.67% of the time, and the reverse 0.00%, while mixed-system training reaches 99.3% on held-out documents of a seen system. A detector trained on dedicated NMT catches an LLM prompted to translate 20 times in 100, against 97 when the same model generates. Across four snapshots (2018–2026), the detector-flagged non-human share rises from 2.13% to 6.20%, and after 2023 the flagged movement is on the translation side: the MT-flagged share climbs from 0.30% to 3.48%, while the LLM-flagged share peaks and falls. These are flagged shares, not validated prevalence: on the only independent Bengali corpus available, which has no translated text, the detector scores 62.7% against 97.3% in-benchmark. The benchmark, checkpoints and scripts will be released.

This paper describes the UTSK26 submission to the WAT2026 Patent Claims Translation/Evaluation Task for the Japanese-to-English (JA-EN) and English-to-Japanese (EN-JA) directions. The task requires participants to annotate translation errors in machine translation (MT) outputs of patent claims and to assign a quality score from 0 to 100 to each translation. For error annotation, we use GEMBA-Patent-Hybrid, which combines an LLM-as-a-Judge method for detecting error spans, categories, and severities using GPT-5.2 with another method for assigning categories and severities to encoder-detected error spans. For quality scoring, we use Patent-Rubric-Checklist, a rubric-based LLM-as-a-Judge method that instructs GPT-5.2 to answer patent-specific checklist items covering five evaluation dimensions and aggregates the answers into a score with predefined weights. On the training data, which consist of the human evaluation results of the WAT2025 task, Patent-Rubric-Checklist achieves the highest Pearson and Spearman correlations with human scores in both directions among the evaluated methods, including COMETKiwi, MetricX-24, GEMBA-DA, and GEMBA-MQM.

This paper describes the Ehime-U system submitted to the Japanese–English Patent Claims Translation/Evaluation Task at WAT 2026. Our system predicts translation errors by providing an LLM with word-level correspondences between source and target claims. We compare six settings that differ in how these correspondences are obtained and whether they are used: four Unbalanced Optimal Transport (UOT)-based alignment methods (Forward, Reverse, AND, and OR), one alignment generated by LLM, and a baseline without alignment. We discuss how the choice of word correspondence affects error prediction. Against the gold reference annotations, under the hyperparameter settings used in this study, providing word correspondences improves error-detection F1 by up to 3.7 points over the no-alignment baseline (0.512 vs. 0.475). The improvement is category-specific: recall increases for alignment-dependent error categories but not for fluency errors. Comparing the two approaches to obtaining correspondences, we find that LLM-based alignment covers substantially more source tokens than the UOT alignments and achieves a higher F1 score than the best UOT setting under the hyperparameter settings used in this study (0.512 vs. 0.486).

We describe GABU, our WAT2026 English-to-Indic multimodal translation system (CITATION), which translates image grounded English captions into Hindi, Bengali, Malayalam, and Odia. The task follows the WAT English-to-Indic multimodal track setup with text only, image captioning, and multimodal tracks (CITATION). Our system builds on the pretrained, instruction-tuned vision-language model google/gemma-4-E4B-it. Each input combines a cropped visual region with the English caption, rendered through a chat template with a script constrained translation instruction. We explore two approaches which includes a zero shot baseline and a fine tuned approach of the model using QLORA (rank 64, alpha 128, 4-bit NF4, double quantization), trained for 3,000 steps due to computational resource constraints, reflecting our aim to adapt the model’s existing instruction-following ability rather than train it from scratch. We evaluate both systems using BLEU and CHRF3 under greedy decoding and report macro averaged scores across the target languages. The fine-tuned model substantially outperforms the zero shot baseline and achieves a BLEU score of 43.97 and 69.99 CHRF3 on the test split, and 40.37 BLEU and 68.25 CHRF3 on the challenge split. These results correspond to improvements of 24.39 BLEU and 18.72 CHRF3 points on the test split, and 16.70 BLEU and 11.90 CHRF3 points on the challenge split.

This paper presents the results and findings of the 2nd Patent Claims Translation/Evaluation Task at WAT2026. Building on the first task, which collected human error annotations for machine-translated patent claims, this year we introduce an evaluation task: given the source claims and the machine translations gathered last year, participants automatically annotate translation errors (spans, categories, and severities) and assign a holistic quality score to each translation. We received submissions from three teams covering the Japanese–English (JA–EN) and English–Japanese (EN–JA) directions. We evaluated the submissions against the organizers’ reference annotations in terms of error-span detection, severity, and quality-score correlation and correlated the results with the design of each participating system. Our analysis highlights two observations. First, giving an LLM explicit word alignments mainly helps it detect errors that hinge on the source–target correspondence (mistranslation and terminology), while leaving target-only fluency errors unaffected. Second, locating errors and producing a good overall quality score behave as largely independent abilities: the system whose score correlated best with the reference detected the fewest errors, because it scored quality with a module separate from its error annotator.