Neural machine translation (NMT) of low-resource languages is often greatly improved through parent-child transfer learning. When high-resource parent and low-resource child languages are closely related, the system benefits from high vocabulary overlap and shared representations. However, systematic surface-level differences in orthography hide many of the rich connections between related languages, even when written in the same script, and prevent knowledge transfer. We present CharLOTTE: a completely automatic and language-agnostic pipeline that overcomes this hurdle without the need of manually crafted rules or external linguistic resources. CharLOTTE collects a dataset of cognates by running word alignment on parallel sentences and filtering the resulting word pairs based on normalized edit distance. The pipeline learns systematic orthographic correspondences from the cognates and applies them to more closely unify the representation of a high-resource language to that of a related low-resource language, thus increasing vocabulary overlap and enhancing knowledge transfer that improves downstream NMT. We evaluate CharLOTTE on four low-resource translation pairs, including one creole language pair, and report significant improvement in translation quality of up to 5.53 BLEU.

The morphological structure of Semitic languages, such as Arabic, is based on non-concatenative roots and templates. This complex word structure used by humans is obscured to neural models that employ traditional tokenization algorithms, such as byte-pair encoding (BPE) (Sennrich et al., 2016; Gage, 1994). In this work, we present and evaluate Semitic Root Encoding (SRE), a tokenization method that represents both concatenative and non-concatenative structures in Semitic words with sequences of root, template stem, and BPE tokens. We apply the method to neural machine translation (NMT) and find that SRE tokenization yields an average increase of 1.15 BLEU over the baseline. SRE tokenization is also robust against generating combinations of roots with template stems that do not occur in nature. Finally, we compare the performance of SRE to tokenization based on non-linguistic root and template structures and tokenization based on stems, providing evidence that NMT models are capable of leveraging tokens based on non-concatenative Semitic morphology.