An HMM Approach with Inherent Model Selection for Sign Language and Gesture Recognition

Sandrine Tornay, Oya Aran, Mathew Magimai Doss


Abstract
HMMs have been the one of the first models to be applied for sign recognition and have become the baseline models due to their success in modeling sequential and multivariate data. Despite the extensive use of HMMs for sign recognition, determining the HMM structure has still remained as a challenge, especially when the number of signs to be modeled is high. In this work, we present a continuous HMM framework for modeling and recognizing isolated signs, which inherently performs model selection to optimize the number of states for each sign separately during recognition. Our experiments on three different datasets, namely, German sign language DGS dataset, Turkish sign language HospiSign dataset and Chalearn14 dataset show that the proposed approach achieves better sign language or gesture recognition systems in comparison to the approach of selecting or presetting the number of HMM states based on k-means, and yields systems that perform competitive to the case where the number of states are determined based on the test set performance.
Anthology ID:
2020.lrec-1.741
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
6049–6056
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.741
DOI:
Bibkey:
Cite (ACL):
Sandrine Tornay, Oya Aran, and Mathew Magimai Doss. 2020. An HMM Approach with Inherent Model Selection for Sign Language and Gesture Recognition. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6049–6056, Marseille, France. European Language Resources Association.
Cite (Informal):
An HMM Approach with Inherent Model Selection for Sign Language and Gesture Recognition (Tornay et al., LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.741.pdf