Greek Sign Language Recognition for the SL-ReDu Learning Platform

Katerina Papadimitriou, Gerasimos Potamianos, Galini Sapountzaki, Theodore Goulas, Eleni Efthimiou, Stavroula-Evita Fotinea, Petros Maragos


Abstract
There has been increasing interest lately in developing education tools for sign language (SL) learning that enable self-assessment and objective evaluation of learners’ SL productions, assisting both students and their instructors. Crucially, such tools require the automatic recognition of SL videos, while operating in a signer-independent fashion and under realistic recording conditions. Here, we present an early version of a Greek Sign Language (GSL) recognizer that satisfies the above requirements, and integrate it within the SL-ReDu learning platform that constitutes a first in GSL with recognition functionality. We develop the recognition module incorporating state-of-the-art deep-learning based visual detection, feature extraction, and classification, designing it to accommodate a medium-size vocabulary of isolated signs and continuously fingerspelled letter sequences. We train the module on a specifically recorded GSL corpus of multiple signers by a web-cam in non-studio conditions, and conduct both multi-signer and signer-independent recognition experiments, reporting high accuracies. Finally, we let student users evaluate the learning platform during GSL production exercises, reporting very satisfactory objective and subjective assessments based on recognition performance and collected questionnaires, respectively.
Anthology ID:
2022.sltat-1.12
Volume:
Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Eleni Efthimiou, Stavroula-Evita Fotinea, Thomas Hanke, John C. McDonald, Dimitar Shterionov, Rosalee Wolfe
Venue:
SLTAT
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
79–84
Language:
URL:
https://aclanthology.org/2022.sltat-1.12
DOI:
Bibkey:
Cite (ACL):
Katerina Papadimitriou, Gerasimos Potamianos, Galini Sapountzaki, Theodore Goulas, Eleni Efthimiou, Stavroula-Evita Fotinea, and Petros Maragos. 2022. Greek Sign Language Recognition for the SL-ReDu Learning Platform. In Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives, pages 79–84, Marseille, France. European Language Resources Association.
Cite (Informal):
Greek Sign Language Recognition for the SL-ReDu Learning Platform (Papadimitriou et al., SLTAT 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.sltat-1.12.pdf