Towards Performance Improvement in Indian Sign Language Recognition

Kinjal Mistree, Devendra Thakor, Brijesh Bhatt


Abstract
Sign language is a complete natural language used by deaf and dumb people. It has its own grammar and it differs with spoken language to a great extent. Since people without hearing and speech impairment lack the knowledge of the sign language, the deaf and dumb people find it difficult to communicate with them. The conception of system that would be able to translate the sign language into text would facilitate understanding of sign language without human interpreter. This paper describes a systematic approach that takes Indian Sign Language (ISL) video as input and converts it into text using frame sequence generator and image augmentation techniques. By incorporating these two concepts, we have increased dataset size and reduced overfitting. It is demonstrated that using simple image manipulation techniques and batch of shifted frames of videos, performance of sign language recognition can be significantly improved. Approach described in this paper achieves 99.57% accuracy on the dynamic gesture dataset of ISL.
Anthology ID:
2020.icon-main.47
Volume:
Proceedings of the 17th International Conference on Natural Language Processing (ICON)
Month:
December
Year:
2020
Address:
Indian Institute of Technology Patna, Patna, India
Editors:
Pushpak Bhattacharyya, Dipti Misra Sharma, Rajeev Sangal
Venue:
ICON
SIG:
Publisher:
NLP Association of India (NLPAI)
Note:
Pages:
349–354
Language:
URL:
https://aclanthology.org/2020.icon-main.47
DOI:
Bibkey:
Cite (ACL):
Kinjal Mistree, Devendra Thakor, and Brijesh Bhatt. 2020. Towards Performance Improvement in Indian Sign Language Recognition. In Proceedings of the 17th International Conference on Natural Language Processing (ICON), pages 349–354, Indian Institute of Technology Patna, Patna, India. NLP Association of India (NLPAI).
Cite (Informal):
Towards Performance Improvement in Indian Sign Language Recognition (Mistree et al., ICON 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.icon-main.47.pdf