Colin Lea
Author directory2026
DiscoSign: Discourse-Aware Text to Sign Language Gloss Translation
Vasileios Baltatzis | Mert Inan | Connor Gillis | Raja Kushalnagar | Lorna Quandt | Leah Findlater | Colin Lea
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Vasileios Baltatzis | Mert Inan | Connor Gillis | Raja Kushalnagar | Lorna Quandt | Leah Findlater | Colin Lea
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Sign language processing systems have traditionally operated at the sentence level, ignoring critical discourse phenomena fundamental to sign language comprehension. We introduce DiscoSign, a computational approach for discourse-aware text to sign language gloss translation grounded in linguistic research. We address three key phenomena within our modular Large Language Model (LLM)-based translation framework: (i) spatial coreference resolution, where entities maintain consistent spatial locations throughout discourse; (ii) Question-Answer Clauses (QACs), pseudocleft structures serving specific discourse functions; and (iii) concept-gloss consistency, ensuring stable mappings between English concepts and American Sign Language (ASL) signs. Traditional translation metrics fail to capture discourse-level quality, so we introduce a suite of novel evaluation metrics designed to assess each dimension of discourse coherence addressed by our framework. Experiments on sentence-level and discourse-level datasets show that our approach for discourse-aware processing significantly improves spatial consistency and entity tracking relative to sentence-only translation, while maintaining competitive single-sentence gloss translation quality. Our work establishes the first systematic framework for discourse-level text to sign language gloss translation with corresponding evaluation methodology.
2024
Hypernetworks for Personalizing ASR to Atypical Speech
Max Müller-Eberstein | Dianna Yee | Karren Yang | Gautam Varma Mantena | Colin Lea
Transactions of the Association for Computational Linguistics, Volume 12
Max Müller-Eberstein | Dianna Yee | Karren Yang | Gautam Varma Mantena | Colin Lea
Transactions of the Association for Computational Linguistics, Volume 12
Parameter-efficient fine-tuning (PEFT) for personalizing automatic speech recognition (ASR) has recently shown promise for adapting general population models to atypical speech. However, these approaches assume a priori knowledge of the atypical speech disorder being adapted for—the diagnosis of which requires expert knowledge that is not always available. Even given this knowledge, data scarcity and high inter-/intra-speaker variability further limit the effectiveness of traditional fine-tuning. To circumvent these challenges, we first identify the minimal set of model parameters required for ASR adaptation. Our analysis of each individual parameter’s effect on adaptation performance allows us to reduce Word Error Rate (WER) by half while adapting 0.03% of all weights. Alleviating the need for cohort-specific models, we next propose the novel use of a meta-learned hypernetwork to generate highly individualized, utterance-level adaptations on-the-fly for a diverse set of atypical speech characteristics. Evaluating adaptation at the global, cohort, and individual-level, we show that hypernetworks generalize better to out-of-distribution speakers, while maintaining an overall relative WER reduction of 75.2% using 0.1% of the full parameter budget.