From Subwords to Pixels: Evaluating Text Representations for Arabic Machine Translation

Abdelrahman Shaheen, Mohamed Abbas, Tereza Truckova, Yova Kementchedjhieva, Bashar Alhafni


Abstract
Subword tokenization underpins modern neural NLP but can be suboptimal for morphologically rich languages with substantial orthographic variation. In this paper, we compare vocabulary-based and vocabulary-free representations for Arabic-to-English machine translation, including subword, byte-level, and pixel-level approaches. Across 35 evaluation sets spanning Modern Standard Arabic (MSA) and 18 dialects, all approaches achieve comparable translation performance with sufficient supervision. While subword models perform best with limited MSA pretraining data, differences diminish with scale and dialect-specific fine-tuning. Pixel-based models further exhibit the greatest robustness to unseen orthographic variation and provide the highest training and inference throughput. We make our code and models publicly available.
Anthology ID:
2026.arabicnlp-main.19
Volume:
Proceedings of The Fourth Arabic Natural Language Processing Conference
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Muhammad Abdul-Mageed, Tamer Elsayed, Houda Bouamor, Samhaa El-Beltagy, Hend Al-Khalifa, Ahmed Abdelali, AbdelRahim Elmadany, Abdellah El Mekki, Kareem Darwish, Sakhar Alkhereyf, Hamzah Luqman, Salam Khalifa, Samar Magdy, Zaid Alyafeai, Injy Hamed, Wajdi Zaghouani, Imed Zitouni
Venue:
ArabicNLP
SIG:
SIGARAB
Publisher:
Association for Computational Linguistics
Note:
Pages:
263–288
Language:
URL:
https://aclanthology.org/2026.arabicnlp-main.19/
DOI:
Bibkey:
Cite (ACL):
Abdelrahman Shaheen, Mohamed Abbas, Tereza Truckova, Yova Kementchedjhieva, and Bashar Alhafni. 2026. From Subwords to Pixels: Evaluating Text Representations for Arabic Machine Translation. In Proceedings of The Fourth Arabic Natural Language Processing Conference, pages 263–288, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
From Subwords to Pixels: Evaluating Text Representations for Arabic Machine Translation (Shaheen et al., ArabicNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.arabicnlp-main.19.pdf