Evaluating Native-Speaker Preferences on Machine Translation and Post-Edits for Five African Languages

Hiba El Oirghi, Tajuddeen Gwadabe, Marine Carpuat


Abstract
Wikipedia editors undertake the task of editing machine translation (MT) outputs in various languages to disseminate multilingual knowledge from English. But are editors doing more than just translating or fixing MT output? To answer this broad question, we constructed a dataset of 4,335 fine-grained annotated parallel pairs of MT translations and human post-edit (HE) translations for five low-resource African languages: Hausa, Igbo, Swahili, Yoruba, and Zulu. We report on our data selection and annotation methodologies as well as findings from the annotated dataset, the most surprising of which is that annotators mostly preferred the MT translations over their HE counterparts for three out of five languages. We analyze the nature of these "fluency breaking" edits and provide recommendations for the MT post-editing workflows in the Wikipedia domain and beyond.
Anthology ID:
2026.africanlp-main.15
Volume:
Proceedings of the 7th Workshop on African Natural Language Processing (AfricaNLP 2026)
Month:
March
Year:
2026
Address:
Rabat, Morocco
Editors:
Everlyn Asiko Chimoto, Constantine Lignos, Shamsuddeen Muhammad, Idris Abdulmumin, Clemencia Siro, David Ifeoluwa Adelani
Venues:
AfricaNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
163–170
Language:
URL:
https://aclanthology.org/2026.africanlp-main.15/
DOI:
Bibkey:
Cite (ACL):
Hiba El Oirghi, Tajuddeen Gwadabe, and Marine Carpuat. 2026. Evaluating Native-Speaker Preferences on Machine Translation and Post-Edits for Five African Languages. In Proceedings of the 7th Workshop on African Natural Language Processing (AfricaNLP 2026), pages 163–170, Rabat, Morocco. Association for Computational Linguistics.
Cite (Informal):
Evaluating Native-Speaker Preferences on Machine Translation and Post-Edits for Five African Languages (El Oirghi et al., AfricaNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.africanlp-main.15.pdf