O reconhecimento contínuo de Língua Brasileira de Sinais (Libras) apresenta desafios relacionados à natureza multimodal da sinalização e à modelagem de dependências espaço-temporais complexas. Este trabalho em andamento busca investigar a adaptação de arquiteturas neurais para o reconhecimento contínuo de Libras utilizando vídeos RGB, keypoints corporais, faciais e manuais, além de anotações linguísticas multi-tier produzidas no ELAN. O método proposto combina extração de características visuais, integração multimodal e modelagem temporal. A avaliação será realizada por meio da métrica Word Error Rate (WER) e estudos de ablação. Espera-se analisar o impacto da multimodalidade e das anotações multicamadas no desempenho de modelos de reconhecimento contínuo de Libras.

This paper addresses the challenges of using sign language within shared mobility services, such as taxis, carpools, or ride-sharing platforms. The use of sign language recognition (SLR) in real-world, confined environments, specifically vehicle interiors remains largely unexplored. To motivate research in this area, we present the In-Car Sign Language (ICSL) dataset for Brazilian Sign Language (Libras), with the long-term goal of improving public transport accessibility for the Deaf and Hard-of-Hearing community. The dataset consists of: (1) high-precision laboratory motion capture (MoCap) data to establish an idealized linguistic baseline and (2) real-world multi-modal in-car recordings captured using a 2D camera and 3D Time-of-Flight sensors. The dataset provides a basis for comparative analyses between synthesized signing avatar animations and recorded real signing interpreter videos, which enable future research into robust “in-the-wild” SLR models and domain adaptation. We describe in detail the use cases, the setup, the data collection protocol, and the metadata structure of the corpus. In total, we recorded a multimodal dataset exceeding 1.5 million frames, comprising the synchronized multimodal streams described above featuring Libras users across various in-car scenarios. The corpus is provided with gloss annotation of lexical signs and non-lexical sign language elements specially designed to support the training and evaluation of deep neural networks for constrained space recognition. In-vehicle signing offers a technically significant example of a constrained, occluded, and non-frontal environment. While recognizing the diverse communication strategies already employed by the Deaf community, identifying automotive-specific limitations provides a useful stepping stone for research into enhancing in-car accessibility and passenger quality of life.