Shared-Encoder Contrastive Grounding for Pose-Based Indian Sign Language Translation

Gyanendra Shrestha, Liting Zhang


Abstract
The WSLP 2026 shared task studies sentence-level translation from Indian Sign Language pose sequences to English, with training on iSign and evaluation on a disjoint collection of source videos. We address two challenges in this setting: unstable pose scale caused by landmark errors and weak grounding between pose inputs and generated text. We introduce robust clip-level normalization and shared-encoder contrastive grounding, which aligns pose and target-text representations in a pretrained T5 encoder during training while retaining standard autoregressive decoding at inference. On a video-held-out iSign development split, contrastive grounding improves chrF from 0.102 to 0.121, while clip-level normalization further improves it to 0.128; the normalization gain is consistent across three matched seeds. A retrieval probe provides complementary evidence of improved pose–text alignment. Our final shared-task submission achieves chrF 0.16 on the official test set. However, input-independent English controls reach chrF 0.15, showing that small differences in the official score are difficult to interpret at the current level of translation accuracy. The source code is available at https://github.com/gyan-shrestha/WSLP-2026-SLT.
Anthology ID:
2026.wslp-1.13
Volume:
Proceedings of the Second Workshop on Sign Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Sanjeet Singh, Abhinav Joshi, Pokala Dattatreya, Keren Artiaga, Khushi Jaydeep Pujara, Tomáš Železný, Ivan Gruber, Jakub Honzík, Sabyasachi Kamila, Facundo Manuel Quiroga, Marek Hrúz, Mohammed Hasanuzzaman, Ashutosh Modi
Venues:
WSLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
132–140
Language:
URL:
https://aclanthology.org/2026.wslp-1.13/
DOI:
Bibkey:
Cite (ACL):
Gyanendra Shrestha and Liting Zhang. 2026. Shared-Encoder Contrastive Grounding for Pose-Based Indian Sign Language Translation. In Proceedings of the Second Workshop on Sign Language Processing, pages 132–140, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Shared-Encoder Contrastive Grounding for Pose-Based Indian Sign Language Translation (Shrestha & Zhang, WSLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.wslp-1.13.pdf