Confident but Wrong: A Constrained Decoding Diagnostic for Low-Resource Automatic Post-Editing

Isuru Wijesiri, Nisansa de Silva, Kavindu Warnakulasuriya, Aloka Fernando, Surangika Ranathunga


Abstract
Automatic Post-Editing (APE) for low-resource languages (LRLs) often fails to improve Machine Translation (MT), and the score alone cannot say why: whether more training would help, or whether the training data is too inconsistent to learn from. We introduce a black-box, inference-time diagnostic that tells these two cases apart without retraining or annotation. It varies an edit-distance penalty 𝜆 that drives the model from free editing towards copying the MT, and reads two signals: (1) the shape of the Translation Edit Rate (TER)-vs-𝜆 curve, U-shaped if edits from the model reduce error and monotonically decreasing if none does; and (2) the ordering of constraint variants that trust model confidence to increasing degrees, which shows whether confidence tracks edit quality. Across decoder-only and encoder–decoder models on English–Sinhala, the diagnostic exposes two failure modes consistent with a heterogeneous post-edit signal as the underlying cause: Binary Collapse, where the model copies the MT or makes off-target edits, and Confident Miscalibration, where the confidence signals we test do not separate useful edits from unnecessary ones. The pattern holds on English–Marathi and English–Tamil, with the failure modes tracking the post-edit distribution rather than MT quality or language family. Beyond diagnosis, the curve shape prescribes a concrete next step for practitioners; in the favorable case, a static constraint yields a free inference-time accuracy gain. We release the first English–Sinhala (∼66k) and a new English–Tamil (∼39k) APE datasets with all code.
Anthology ID:
2026.findings-emnlp.1079
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
22838–22866
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1079/
DOI:
Bibkey:
Cite (ACL):
Isuru Wijesiri, Nisansa de Silva, Kavindu Warnakulasuriya, Aloka Fernando, and Surangika Ranathunga. 2026. Confident but Wrong: A Constrained Decoding Diagnostic for Low-Resource Automatic Post-Editing. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 22838–22866, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Confident but Wrong: A Constrained Decoding Diagnostic for Low-Resource Automatic Post-Editing (Wijesiri et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1079.pdf