Özge Can Özel
Author directory2026
Cross-Lingual Transfer of Self-Supervised Representations for Sign Language Translation: A Case Study for Turkish Sign Language
Özge Can Özel | Saliha Nur Şahin | Murat Saraclar
Proceedings of the Second Workshop on Sign Language Processing
Özge Can Özel | Saliha Nur Şahin | Murat Saraclar
Proceedings of the Second Workshop on Sign Language Processing
Sign Language Translation (SLT) is especially difficult for low-resource languages such as Turkish Sign Language (TID), where aligned sign-text data are scarce. Self-supervised encoders pretrained on large sign corpora can reduce this dependence, but existing models are developed and evaluated almost exclusively on high-resource sign languages. One such model is SHuBERT (Sign Hidden-Unit BERT), a publicly released self-supervised contextual representation model trained on American Sign Language (ASL). We ask whether SHuBERT can be adapted to TID translation on the open-domain BUTID benchmark. Pretrained SHuBERT is incorporated into an encoder-decoder based translation model and evaluated under both frozen and fine-tuned settings. This is compared against two references trained under the same recipe: a pose baseline, and a raw-feature setting using DINOv2-encoded hand/face crops and pose. All variants outperform the pose baseline, frozen SHuBERT performs on par with raw-feature setting, and fine-tuning yields the strongest result, reaching 7.15 BLEU-4 on the test set (+1.62 over the pose baseline, +0.77 over the raw-feature setting). Our results suggest that reusing a high resource, ASL-pretrained self-supervised encoder can benefit low-resource SLT, but this benefit emerges through fine-tuning rather than frozen transfer alone.