Towards Precise Localization of Critical Errors in Machine Translation

Dahyun Jung, Sugyeong Eo, Heuiseok Lim


Abstract
The advent of large language models has experienced a remarkable improvement in the field of machine translation. However, machine translation is still vulnerable to critical meaning deviations, which may incur catastrophic issues in social or ethical contexts. In particular, existing critical error detection primarily focuses on identifying sentence-level errors, leaving the precise localization of such errors within the sentence unaddressed. In this paper, we introduce a new task, word-level critical error detection (WCED), to detect critical errors at a fine-grained level in machine translation sentences. The task aims to identify the parts of a machine translation that contain catastrophic meaning distortions. We hypothesize that the ability to determine errors at the sentence level will positively influence the detection of more granular errors. We propose a sentence-level error detection module to predict which words in a sentence have critical errors. Experimental results demonstrate that our method outperforms existing methodologies and LLM in En-De, Zh-En, En-Ru, and En-Ko. Our method is helpful for determining the fine-grained location of errors. We hope that such studies will improve the capacity to address critical errors adeptly.
Anthology ID:
2024.findings-acl.177
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3000–3012
Language:
URL:
https://aclanthology.org/2024.findings-acl.177
DOI:
Bibkey:
Cite (ACL):
Dahyun Jung, Sugyeong Eo, and Heuiseok Lim. 2024. Towards Precise Localization of Critical Errors in Machine Translation. In Findings of the Association for Computational Linguistics ACL 2024, pages 3000–3012, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
Towards Precise Localization of Critical Errors in Machine Translation (Jung et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.177.pdf