ISLiM: A Compact End-to-End Sign Language Translation Framework

Shoaib Ahmed, Dipti Mishra


Abstract
Sign Language Translation (SLT) is a key accessibility technology for Deaf communities. Although recent years have seen rapid progress in SLT, Indian Sign Language (ISL) and other sign languages of under-resourced regions remain under-researched and overshadowed by sign languages with a longer research history and stronger infrastructure. In this direction, we present ISLiM (ISL in Miniature), a compact end-to-end keypoint-based gloss-free SLT architecture that maps body-pose landmark sequences directly to spoken-language text. At its core, a stack of temporal convolutional layers captures low-level pose motion patterns, a Transformer-based encoder models higher-order kinematic interactions and sign-level temporal ordering, and a pretrained T5-small decoder with a prior grasp of the target language performs the semantic heavy-lifting of generating fluent text. We further explore complementary loss functions that sustain training on small datasets, and alternative qualitative evaluation metrics tailored to the particular challenges of SLT. ISLiM achieves a BLEU-4 score of 5.85 on iSign, establishing state-of-the-art results among reported keypoint-based models on the only large-scale ISL dataset. We also show promising transferability to German Sign Language on Phoenix-2014-T through modular decoder swap, pointing toward cross-lingual extensibility where both sign and spoken languages can be swapped independently. A preliminary mobile deployment study confirms that the full keypoint-based model runs on a consumer Android smartphone with ~384ms mean inference latency on CPU, demonstrating a practical path toward offline, accessible assistive tools for the Deaf community in India and the world. Code is available at https://github.com/ahmedshoaib/ISLiM
Anthology ID:
2026.wslp-1.3
Volume:
Proceedings of the Second Workshop on Sign Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Sanjeet Singh, Abhinav Joshi, Pokala Dattatreya, Keren Artiaga, Khushi Jaydeep Pujara, Tomáš Železný, Ivan Gruber, Jakub Honzík, Sabyasachi Kamila, Facundo Manuel Quiroga, Marek Hrúz, Mohammed Hasanuzzaman, Ashutosh Modi
Venues:
WSLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
18–31
Language:
URL:
https://aclanthology.org/2026.wslp-1.3/
DOI:
Bibkey:
Cite (ACL):
Shoaib Ahmed and Dipti Mishra. 2026. ISLiM: A Compact End-to-End Sign Language Translation Framework. In Proceedings of the Second Workshop on Sign Language Processing, pages 18–31, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
ISLiM: A Compact End-to-End Sign Language Translation Framework (Ahmed & Mishra, WSLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.wslp-1.3.pdf