Evaluation of Manual and Non-manual Components for Sign Language Recognition

Medet Mukushev, Arman Sabyrov, Alfarabi Imashev, Kenessary Koishybay, Vadim Kimmelman, Anara Sandygulova


Abstract
The motivation behind this work lies in the need to differentiate between similar signs that differ in non-manual components present in any sign. To this end, we recorded full sentences signed by five native signers and extracted 5200 isolated sign samples of twenty frequently used signs in Kazakh-Russian Sign Language (K-RSL), which have similar manual components but differ in non-manual components (i.e. facial expressions, eyebrow height, mouth, and head orientation). We conducted a series of evaluations in order to investigate whether non-manual components would improve sign’s recognition accuracy. Among standard machine learning approaches, Logistic Regression produced the best results, 78.2% of accuracy for dataset with 20 signs and 77.9% of accuracy for dataset with 2 classes (statement vs question). Dataset can be downloaded from the following website: https://krslproject.github.io/krsl20/
Anthology ID:
2020.lrec-1.745
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
6073–6078
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.745
DOI:
Bibkey:
Cite (ACL):
Medet Mukushev, Arman Sabyrov, Alfarabi Imashev, Kenessary Koishybay, Vadim Kimmelman, and Anara Sandygulova. 2020. Evaluation of Manual and Non-manual Components for Sign Language Recognition. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6073–6078, Marseille, France. European Language Resources Association.
Cite (Informal):
Evaluation of Manual and Non-manual Components for Sign Language Recognition (Mukushev et al., LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.745.pdf