Jacky Li


2023

pdf bib
Sign Language Recognition and Translation: A Multi-Modal Approach Using Computer Vision and Natural Language Processing
Jacky Li | Jaren Gerdes | James Gojit | Austin Tao | Samyak Katke | Kate Nguyen | Benyamin Ahmadnia
Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing

Sign-to-Text (S2T) is a hand gesture recognition program in the American Sign Language (ASL) domain. The primary objective of S2T is to classify standard ASL alphabets and custom signs and convert the classifications into a stream of text using neural networks. This paper addresses the shortcomings of pure Computer Vision techniques and applies Natural Language Processing (NLP) as an additional layer of complexity to increase S2T’s robustness.