Machine translation shared tasks publish several automatic metrics and rank systems on one of them, assuming the choice is inconsequential. We test this assumption on the WMT 2025 and WMT 2026 Low-Resource Indic Language Translation tasks, where pre-trained multilingual encoders are unavailable for several target scripts and languages. As no human gold standard evaluation exists at scale, we adapt the Quantified Reproducibility Assessment (QRA) framework across 20 WMT26 and 14 WMT25 directions, to measure agreement between six automatic metrics and two LLM judges as independent measuring instruments. We show that QRA’s Type I measure fails under this repurposing when instruments sit on different scales, and suggest a rank-based repair. Using a script-controlled natural experiment (Manipuri evaluated in both Bengali and Meitei Mayek) and a null-output validity test, we observe that neural metrics inflate scores for text in scripts outside the encoder’s coverage and compress it into a narrow discriminative band. Switching to Meitei Mayek raises BERTScore and COMET by 21.6 and 23.3 points, respectively, though language and content are unchanged, and no metric preserves system ranking; on degenerate output the two retain 0.81 and 0.61 points on English-to-Indic directions. The effect reverses in the opposite direction. Metric choice should follow script coverage, not language identity. Lacking human judgments, our claims concern instrument behaviour, not human preference.

We describe team PANINI’s submission to the WMT 2026 Shared Task on Low-Resource Indic Language Translation. We submit primary systems for English↔Manipuri (Meiteilon) in both the Bengali (mni Beng) and Meitei Mayek (mni Mtei) scripts, covering all four directions. Rather than one shared multilingual adapter, we train four independent rank-stabilized LoRA (rsLoRA) adapters over IndicTrans2-1B, one per direction. Each adapter is trained on the authentic bitext plus three filtered synthetic streams the model generates for itself: back-translation, forward translation, and iterative pseudo-labelling, with subword regularisation and a script-aware processing pipeline. Our system obtains a COMET score of 92.31 and a BERTScore of 98.85 on English↔Manipuri (Meitei Mayek), the best in the direction. On the into-English directions, it reaches a COMET of 82.90 for mni Beng↔en and 80.20 for mniMtei↔en. These results show that per-direction adapters combined with self-generated synthetic data are effective even when authentic bitext is limited to a few thousand sentence pairs, and that a script-aware processing pipeline is necessary to produce well-formed output in both Manipuri scripts. The code and resources are released on GitHub.