Using Neural Machine Translation Methods for Sign Language Translation

Galina Angelova, Eleftherios Avramidis, Sebastian Möller


Abstract
We examine methods and techniques, proven to be helpful for the text-to-text translation of spoken languages in the context of gloss-to-text translation systems, where the glosses are the written representation of the signs. We present one of the first works that include experiments on both parallel corpora of the German Sign Language (PHOENIX14T and the Public DGS Corpus). We experiment with two NMT architectures with optimization of their hyperparameters, several tokenization methods and two data augmentation techniques (back-translation and paraphrasing). Through our investigation we achieve a substantial improvement of 5.0 and 2.2 BLEU scores for the models trained on the two corpora respectively. Our RNN models outperform our Transformer models, and the segmentation method we achieve best results with is BPE, whereas back-translation and paraphrasing lead to minor but not significant improvements.
Anthology ID:
2022.acl-srw.21
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Samuel Louvan, Andrea Madotto, Brielen Madureira
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
273–284
Language:
URL:
https://aclanthology.org/2022.acl-srw.21
DOI:
10.18653/v1/2022.acl-srw.21
Bibkey:
Cite (ACL):
Galina Angelova, Eleftherios Avramidis, and Sebastian Möller. 2022. Using Neural Machine Translation Methods for Sign Language Translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 273–284, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Using Neural Machine Translation Methods for Sign Language Translation (Angelova et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-srw.21.pdf
Code
 dfki-signlanguage/gloss-to-text-sign-language-translation
Data
PHOENIX14TRWTH-PHOENIX-Weather 2014 T