Improving and Extending Continuous Sign Language Recognition: Taking Iconicity and Spatial Language into account

Valentin Belissen, Michèle Gouiffès, Annelies Braffort


Abstract
In a lot of recent research, attention has been drawn to recognizing sequences of lexical signs in continuous Sign Language corpora, often artificial. However, as SLs are structured through the use of space and iconicity, focusing on lexicon only prevents the field of Continuous Sign Language Recognition (CSLR) from extending to Sign Language Understanding and Translation. In this article, we propose a new formulation of the CSLR problem and discuss the possibility of recognizing higher-level linguistic structures in SL videos, like classifier constructions. These structures show much more variability than lexical signs, and are fundamentally different than them in the sense that form and meaning can not be disentangled. Building on the recently published French Sign Language corpus Dicta-Sign-LSF-v2, we discuss the performance and relevance of a simple recurrent neural network trained to recognize illustrative structures.
Anthology ID:
2020.signlang-1.2
Volume:
Proceedings of the LREC2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Eleni Efthimiou, Stavroula-Evita Fotinea, Thomas Hanke, Julie A. Hochgesang, Jette Kristoffersen, Johanna Mesch
Venue:
SignLang
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
7–12
Language:
English
URL:
https://aclanthology.org/2020.signlang-1.2
DOI:
Bibkey:
Cite (ACL):
Valentin Belissen, Michèle Gouiffès, and Annelies Braffort. 2020. Improving and Extending Continuous Sign Language Recognition: Taking Iconicity and Spatial Language into account. In Proceedings of the LREC2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives, pages 7–12, Marseille, France. European Language Resources Association (ELRA).
Cite (Informal):
Improving and Extending Continuous Sign Language Recognition: Taking Iconicity and Spatial Language into account (Belissen et al., SignLang 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.signlang-1.2.pdf