Extracting translation pairs from social network content

Matthias Eck, Yuri Zemlyanskiy, Joy Zhang, Alex Waibel


Abstract
We introduce two methods to collect additional training data for statistical machine translation systems from public social network content. The first method identifies multilingual content where the author self-translated their own post to reach additional friends, fans or customers. Once identified, we can split the post in the language segments and extract translation pairs from this content. The second methods considers web links (URLs) that users add as part of their post to point the reader to a video, article or website. If the same URL is shared from different language users, there is a chance they might give the same comment in their respective language. We use a support vector machine (SVM) as a classifier to identify true translations from all candidate pairs. We collected additional translation pairs using both methods for the language pairs Spanish-English and Portuguese-English. Testing the collected data as additional training data for statistical machine translations on in-domain test sets resulted in very significant improvements of up to 5 BLEU.
Anthology ID:
2014.iwslt-papers.7
Volume:
Proceedings of the 11th International Workshop on Spoken Language Translation: Papers
Month:
December 4-5
Year:
2014
Address:
Lake Tahoe, California
Editors:
Marcello Federico, Sebastian Stüker, François Yvon
Venue:
IWSLT
SIG:
SIGSLT
Publisher:
Note:
Pages:
200–205
Language:
URL:
https://aclanthology.org/2014.iwslt-papers.7
DOI:
Bibkey:
Cite (ACL):
Matthias Eck, Yuri Zemlyanskiy, Joy Zhang, and Alex Waibel. 2014. Extracting translation pairs from social network content. In Proceedings of the 11th International Workshop on Spoken Language Translation: Papers, pages 200–205, Lake Tahoe, California.
Cite (Informal):
Extracting translation pairs from social network content (Eck et al., IWSLT 2014)
Copy Citation:
PDF:
https://aclanthology.org/2014.iwslt-papers.7.pdf