Semi-supervised transliteration mining from parallel and comparable corpora

Walid Aransa, Holger Schwenk, Loic Barrault


Abstract
Transliteration is the process of writing a word (mainly proper noun) from one language in the alphabet of another language. This process requires mapping the pronunciation of the word from the source language to the closest possible pronunciation in the target language. In this paper we introduce a new semi-supervised transliteration mining method for parallel and comparable corpora. The method is mainly based on a new suggested Three Levels of Similarity (TLS) scores to extract the transliteration pairs. The first level calculates the similarity of of all vowel letters and consonants letters. The second level calculates the similarity of long vowels and vowel letters at beginning and end position of the words and consonants letters. The third level calculates the similarity consonants letters only. We applied our method on Arabic-English parallel and comparable corpora. We evaluated the extracted transliteration pairs using a statistical based transliteration system. This system is built using letters instead or words as tokens. The transliteration system achieves an accuracy of 0.50 and a mean F-score 0.8958 when trained on transliteration pairs extracted from a parallel corpus. The accuracy is 0.30 and the mean F-score 0.84 when we used instead a comparable corpus to automatically extract the transliteration pairs. This shows that the proposed semi-supervised transliteration mining algorithm is effective and can be applied to other language pairs. We also evaluated two segmentation techniques and reported the impact on the transliteration performance.
Anthology ID:
2012.iwslt-papers.6
Volume:
Proceedings of the 9th International Workshop on Spoken Language Translation: Papers
Month:
December 6-7
Year:
2012
Address:
Hong Kong, Table of contents
Venue:
IWSLT
SIG:
SIGSLT
Publisher:
Note:
Pages:
185–192
Language:
URL:
https://aclanthology.org/2012.iwslt-papers.6
DOI:
Bibkey:
Cite (ACL):
Walid Aransa, Holger Schwenk, and Loic Barrault. 2012. Semi-supervised transliteration mining from parallel and comparable corpora. In Proceedings of the 9th International Workshop on Spoken Language Translation: Papers, pages 185–192, Hong Kong, Table of contents.
Cite (Informal):
Semi-supervised transliteration mining from parallel and comparable corpora (Aransa et al., IWSLT 2012)
Copy Citation:
PDF:
https://aclanthology.org/2012.iwslt-papers.6.pdf