A Dataset for Linguistic Understanding, Visual Evaluation, and Recognition of Sign Languages: The K-RSL

Alfarabi Imashev, Medet Mukushev, Vadim Kimmelman, Anara Sandygulova


Abstract
The paper presents the first dataset that aims to serve interdisciplinary purposes for the utility of computer vision community and sign language linguistics. To date, a majority of Sign Language Recognition (SLR) approaches focus on recognising sign language as a manual gesture recognition problem. However, signers use other articulators: facial expressions, head and body position and movement to convey linguistic information. Given the important role of non-manual markers, this paper proposes a dataset and presents a use case to stress the importance of including non-manual features to improve the recognition accuracy of signs. To the best of our knowledge no prior publicly available dataset exists that explicitly focuses on non-manual components responsible for the grammar of sign languages. To this end, the proposed dataset contains 28250 videos of signs of high resolution and quality, with annotation of manual and non-manual components. We conducted a series of evaluations in order to investigate whether non-manual components would improve signs’ recognition accuracy. We release the dataset to encourage SLR researchers and help advance current progress in this area toward real-time sign language interpretation. Our dataset will be made publicly available at https://krslproject.github.io/krsl-corpus
Anthology ID:
2020.conll-1.51
Volume:
Proceedings of the 24th Conference on Computational Natural Language Learning
Month:
November
Year:
2020
Address:
Online
Venue:
CoNLL
SIG:
SIGNLL
Publisher:
Association for Computational Linguistics
Note:
Pages:
631–640
Language:
URL:
https://aclanthology.org/2020.conll-1.51
DOI:
10.18653/v1/2020.conll-1.51
Bibkey:
Cite (ACL):
Alfarabi Imashev, Medet Mukushev, Vadim Kimmelman, and Anara Sandygulova. 2020. A Dataset for Linguistic Understanding, Visual Evaluation, and Recognition of Sign Languages: The K-RSL. In Proceedings of the 24th Conference on Computational Natural Language Learning, pages 631–640, Online. Association for Computational Linguistics.
Cite (Informal):
A Dataset for Linguistic Understanding, Visual Evaluation, and Recognition of Sign Languages: The K-RSL (Imashev et al., CoNLL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.conll-1.51.pdf