SignaFusion@WSLP 2026: Gated Vision–Language Adjudication and Multi-Expert Fusion for Isolated Sign Language Recognition

Bolisetty Samanvitha, Ojas Srivastava, Pruthwik Mishra, Praveen Kumar Chandaliya


Abstract
We present our submissions to the WSLP 2026 Isolated Sign Language Recognition shared task. The task requires predicting the correct English gloss from a short Indian Sign Language video.

Our approach first establishes a strong primary prediction using vision features and identifies uncertain cases using model disagreement and prediction confidence. Rather than allowing all available experts to influence every prediction, we selectively invoke auxiliary models for difficult cases and incorporate their evidence through constrained late-fusion strategies. This enables complementary appearance, pose, and vision-language information to be used when the primary prediction is uncertain while preserving reliable predictions for confident cases. We introduce a disagreement-gated Qwen2.5-VL model as an auxiliary decision-making module and further incorporate complementary appearance and pose experts through several constrained fusion strategies.

The final system combines multiple fusion variants using reciprocal rank fusion and achieves 66.16% Top-1 accuracy, 72.43% Top-5 accuracy, and 73.19% Top-10 accuracy on the blind test set
Anthology ID:
2026.wslp-1.23
Volume:
Proceedings of the Second Workshop on Sign Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Sanjeet Singh, Abhinav Joshi, Pokala Dattatreya, Keren Artiaga, Khushi Jaydeep Pujara, Tomáš Železný, Ivan Gruber, Jakub Honzík, Sabyasachi Kamila, Facundo Manuel Quiroga, Marek Hrúz, Mohammed Hasanuzzaman, Ashutosh Modi
Venues:
WSLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
237–242
Language:
URL:
https://aclanthology.org/2026.wslp-1.23/
DOI:
Bibkey:
Cite (ACL):
Bolisetty Samanvitha, Ojas Srivastava, Pruthwik Mishra, and Praveen Kumar Chandaliya. 2026. SignaFusion@WSLP 2026: Gated Vision–Language Adjudication and Multi-Expert Fusion for Isolated Sign Language Recognition. In Proceedings of the Second Workshop on Sign Language Processing, pages 237–242, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
SignaFusion@WSLP 2026: Gated Vision–Language Adjudication and Multi-Expert Fusion for Isolated Sign Language Recognition (Samanvitha et al., WSLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.wslp-1.23.pdf