Dipti Mishra

Author directory

2026

Sign Language Translation (SLT) is a key accessibility technology for Deaf communities. Although recent years have seen rapid progress in SLT, Indian Sign Language (ISL) and other sign languages of under-resourced regions remain under-researched and overshadowed by sign languages with a longer research history and stronger infrastructure. In this direction, we present ISLiM (ISL in Miniature), a compact end-to-end keypoint-based gloss-free SLT architecture that maps body-pose landmark sequences directly to spoken-language text. At its core, a stack of temporal convolutional layers captures low-level pose motion patterns, a Transformer-based encoder models higher-order kinematic interactions and sign-level temporal ordering, and a pretrained T5-small decoder with a prior grasp of the target language performs the semantic heavy-lifting of generating fluent text. We further explore complementary loss functions that sustain training on small datasets, and alternative qualitative evaluation metrics tailored to the particular challenges of SLT. ISLiM achieves a BLEU-4 score of 5.85 on iSign, establishing state-of-the-art results among reported keypoint-based models on the only large-scale ISL dataset. We also show promising transferability to German Sign Language on Phoenix-2014-T through modular decoder swap, pointing toward cross-lingual extensibility where both sign and spoken languages can be swapped independently. A preliminary mobile deployment study confirms that the full keypoint-based model runs on a consumer Android smartphone with ~384ms mean inference latency on CPU, demonstrating a practical path toward offline, accessible assistive tools for the Deaf community in India and the world. Code is available at https://github.com/ahmedshoaib/ISLiM