Translation Deserves Better: Analyzing Translation Artifacts in Cross-lingual Visual Question Answering

ChaeHun Park, Koanho Lee, Hyesu Lim, Jaeseok Kim, Junmo Park, Yu-Jung Heo, Du-Seong Chang, Jaegul Choo


Abstract
Building a reliable visual question answering (VQA) system across different languages is a challenging problem, primarily due to the lack of abundant samples for training. To address this challenge, recent studies have employed machine translation systems for the cross-lingual VQA task. This involves translating the evaluation samples into a source language (usually English) and using monolingual models (i.e., translate-test). However, our analysis reveals that translated texts contain unique characteristics distinct from human-written ones, referred to as translation artifacts. We find that these artifacts can significantly affect the models, confirmed by extensive experiments across diverse models, languages, and translation processes. In light of this, we present a simple data augmentation strategy that can alleviate the adverse impacts of translation artifacts.
Anthology ID:
2024.findings-acl.308
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5193–5221
Language:
URL:
https://aclanthology.org/2024.findings-acl.308
DOI:
10.18653/v1/2024.findings-acl.308
Bibkey:
Cite (ACL):
ChaeHun Park, Koanho Lee, Hyesu Lim, Jaeseok Kim, Junmo Park, Yu-Jung Heo, Du-Seong Chang, and Jaegul Choo. 2024. Translation Deserves Better: Analyzing Translation Artifacts in Cross-lingual Visual Question Answering. In Findings of the Association for Computational Linguistics ACL 2024, pages 5193–5221, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
Translation Deserves Better: Analyzing Translation Artifacts in Cross-lingual Visual Question Answering (Park et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.308.pdf