Raja Kushalnagar
Author directory2026
Including Non-Manual Markers in Studies of American Sign Language Dialogue
Jillian Stebbins | Amber Hamilton | Emily Nover | Stephanie Patterson | Allison Fitch | Patrick Boudreault | Raja Kushalnagar | Cecilia O Alm
Proceedings of the 1st Workshop on Multimodal Interaction in Face-to-Face Dialogue
Jillian Stebbins | Amber Hamilton | Emily Nover | Stephanie Patterson | Allison Fitch | Patrick Boudreault | Raja Kushalnagar | Cecilia O Alm
Proceedings of the 1st Workshop on Multimodal Interaction in Face-to-Face Dialogue
Language technology resources are limited for signed languages. In order to improve these technologies, large amounts of labeled language data stored in annotated corpora are necessary. Many existing annotated corpora in American Sign Language (ASL) do not include non-manual markers (NMMs), i.e., language features that do not involve the hands. This work underscores the importance of including NMMs in sign language research by describing their plurifunctional nature in signed conversation, specifying their link to sign language prosody, examining the current state of ASL corpus research, and recommending a systematized method of annotation which captures the critical but complex variations in NMM production in ASL.
DiscoSign: Discourse-Aware Text to Sign Language Gloss Translation
Vasileios Baltatzis | Mert Inan | Connor Gillis | Raja Kushalnagar | Lorna Quandt | Leah Findlater | Colin Lea
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Vasileios Baltatzis | Mert Inan | Connor Gillis | Raja Kushalnagar | Lorna Quandt | Leah Findlater | Colin Lea
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Sign language processing systems have traditionally operated at the sentence level, ignoring critical discourse phenomena fundamental to sign language comprehension. We introduce DiscoSign, a computational approach for discourse-aware text to sign language gloss translation grounded in linguistic research. We address three key phenomena within our modular Large Language Model (LLM)-based translation framework: (i) spatial coreference resolution, where entities maintain consistent spatial locations throughout discourse; (ii) Question-Answer Clauses (QACs), pseudocleft structures serving specific discourse functions; and (iii) concept-gloss consistency, ensuring stable mappings between English concepts and American Sign Language (ASL) signs. Traditional translation metrics fail to capture discourse-level quality, so we introduce a suite of novel evaluation metrics designed to assess each dimension of discourse coherence addressed by our framework. Experiments on sentence-level and discourse-level datasets show that our approach for discourse-aware processing significantly improves spatial consistency and entity tracking relative to sentence-only translation, while maintaining competitive single-sentence gloss translation quality. Our work establishes the first systematic framework for discourse-level text to sign language gloss translation with corresponding evaluation methodology.
2022
Applying the Stereotype Content Model to assess disability bias in popular pre-trained NLP models underlying AI-based assistive technologies
Brienna Herold | James Waller | Raja Kushalnagar
Ninth Workshop on Speech and Language Processing for Assistive Technologies (SLPAT-2022)
Brienna Herold | James Waller | Raja Kushalnagar
Ninth Workshop on Speech and Language Processing for Assistive Technologies (SLPAT-2022)
Stereotypes are a positive or negative, generalized, and often widely shared belief about the attributes of certain groups of people, such as people with sensory disabilities. If stereotypes manifest in assistive technologies used by deaf or blind people, they can harm the user in a number of ways, especially considering the vulnerable nature of the target population. AI models underlying assistive technologies have been shown to contain biased stereotypes, including racial, gender, and disability biases. We build on this work to present a psychology-based stereotype assessment of the representation of disability, deafness, and blindness in BERT using the Stereotype Content Model. We show that BERT contains disability bias, and that this bias differs along established stereotype dimensions.