HandReader: Advanced Techniques for Efficient Fingerspelling Recognition

Pavel Korotaev, Petr Surovtsev, Alexander Kapitanov, Karina Kvanchiani, Alexander Nagaev


Abstract
Fingerspelling is a significant component of Sign Language (SL), allowing the interpretation of proper names, characterized by fast hand movements during signing. Although previous works on fingerspelling recognition have focused on processing the temporal dimension of videos, there remains room for improving the accuracy of these approaches. This paper introduces HandReader, a group of three architectures designed to address the fingerspelling recognition task. HandReaderRGB employs the novel Temporal Shift-Adaptive Module (TSAM) to process RGB features from videos of varying lengths while preserving important sequential information. HandReaderKP is built on the proposed Temporal Pose Encoder (TPE) operating on keypoints as tensors. Such keypoint composition in a batch allows the encoder to pass them through 2D and 3D convolution layers, utilizing temporal and spatial information and accumulating keypoint coordinates. We also introduce HandReaderRGB+KP — architecture with a joint encoder to benefit from RGB and keypoint modalities. Each HandReader model possesses distinct advantages: on the ChicagoFSWild and ChicagoFSWild+ datasets, HandReader models achieve the best results among methods trained on a single training set, including a new state-of-the-art result on ChicagoFSWild+.
Anthology ID:
2026.wslp-1.19
Volume:
Proceedings of the Second Workshop on Sign Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Sanjeet Singh, Abhinav Joshi, Pokala Dattatreya, Keren Artiaga, Khushi Jaydeep Pujara, Tomáš Železný, Ivan Gruber, Jakub Honzík, Sabyasachi Kamila, Facundo Manuel Quiroga, Marek Hrúz, Mohammed Hasanuzzaman, Ashutosh Modi
Venues:
WSLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
193–202
Language:
URL:
https://aclanthology.org/2026.wslp-1.19/
DOI:
Bibkey:
Cite (ACL):
Pavel Korotaev, Petr Surovtsev, Alexander Kapitanov, Karina Kvanchiani, and Alexander Nagaev. 2026. HandReader: Advanced Techniques for Efficient Fingerspelling Recognition. In Proceedings of the Second Workshop on Sign Language Processing, pages 193–202, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
HandReader: Advanced Techniques for Efficient Fingerspelling Recognition (Korotaev et al., WSLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.wslp-1.19.pdf