Swar Desai

Author directory

2026

Automatic Speech Recognition (ASR) for railway announcements is challenging due to noisy environments and dense, entity-rich content. We introduce RailVaani, a low-latency bilingual (Hindi-English) ASR pipeline for Indian railway public address (PA) announcements, benchmarked on RailKosh – the first bilingual annotated corpus of real railway PA audio (1,513 clips: 1,000 English, 513 Hindi; 53 stations; gold transcripts and entity labels). We evaluate ASR baselines and their LoRA fine-tuned variants with database-aware (DB-aware) post-processing (PP), using WER and task score (mean of train number accuracy, platform accuracy, and announcement-type macro-F1), as task score is the more passenger-relevant metric for this domain. Our key findings: (i) on English PA, our deployed system (Whisper Small + LoRA + PP) achieves 69.8% task score at a real-time factor (RTF) of 0.75; a larger variant (Qwen3-ASR-0.6B + LoRA + PP) matches Google STT (79.7% vs. 79.3%) at RTF=1.37; (ii) on Hindi PA, our deployed system (collabora/whisper-small-hindi + LoRA + PP) achieves 60.7% task score at RTF=0.83, surpassing Whisper Large v3 (43.5%) with a 6x smaller model; and (iii) a user study (N=63; SUS-style median=90.0) indicates strong perceived usability. Both deployed systems run faster than real-time on CPU, enabling deployment without GPU infrastructure. RailVaani, RailKosh, and the RailKosh annotation toolkit are fully open-source.