Functional Data Analysis of Non-manual Marking of Questions in Kazakh-Russian Sign Language

Anna Kuznetsova, Alfarabi Imashev, Medet Mukushev, Anara Sandygulova, Vadim Kimmelman


Abstract
This paper is a continuation of Kuznetsova et al. (2021), which described non-manual markers of polar and wh-questions in comparison with statements in an NLP dataset of Kazakh-Russian Sign Language (KRSL) using Computer Vision. One of the limitations of the previous work was the distortion of the 3D face landmarks when the head was rotated. The proposed solution was to train a simple linear regression model to predict the distortion and then subtract it from the original output. We improve this technique with a multilayer perceptron. Another limitation that we intend to address in this paper is the discrete analysis of the continuous movement of non-manuals. In Kuznetsova et al. (2021) we averaged the value of the non-manual over its scope for statistical analysis. To preserve information on the shape of the movement, in this study we use a statistical tool that is often used in speech research, Functional Data Analysis, specifically Functional PCA.
Anthology ID:
2022.signlang-1.19
Volume:
Proceedings of the LREC2022 10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Eleni Efthimiou, Stavroula-Evita Fotinea, Thomas Hanke, Julie A. Hochgesang, Jette Kristoffersen, Johanna Mesch, Marc Schulder
Venue:
SignLang
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
124–131
Language:
URL:
https://aclanthology.org/2022.signlang-1.19
DOI:
Bibkey:
Cite (ACL):
Anna Kuznetsova, Alfarabi Imashev, Medet Mukushev, Anara Sandygulova, and Vadim Kimmelman. 2022. Functional Data Analysis of Non-manual Marking of Questions in Kazakh-Russian Sign Language. In Proceedings of the LREC2022 10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources, pages 124–131, Marseille, France. European Language Resources Association.
Cite (Informal):
Functional Data Analysis of Non-manual Marking of Questions in Kazakh-Russian Sign Language (Kuznetsova et al., SignLang 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.signlang-1.19.pdf