NE-MultiSpeech: A Multilingual Speech Corpus and ASR Benchmark for Northeast Indian Languages

Badal Nyalang, Angana Borah


Abstract
NE-MultiSpeech is a large-scale transcribed multilingual speech corpus covering eight indigenous and regional languages of Northeast India: Khasi, Garo, Mizo, Nagamese, Kokborok, Assamese, Wancho, and Chakma. The corpus combines 124 hours of newly collected, speaker-compensated proprietary recordings across five languages with curated subsets from the Vaani dataset (Pulikodan et al., 2026), totalling approximately 267 hours of transcribed speech across three language families. We fine-tune Whisper-medium on NE-MultiSpeech and establish a systematic automatic speech recognition (ASR) benchmark across all eight languages, achieving an overall word error rate (WER) of 35.53% over 8,099 test utterances jointly, outperforming both Whisper-large-v3 zero-shot (136–194% WER) and MMS-1B zero-shot (42–116% WER) by a large margin. We validate our proxy-language-token strategy for unsupported languages with per-language ablations on four Tibeto-Burman languages, confirming that family-matched proxies consistently outperform mismatched alternatives. We additionally introduce NE-SpeechEmbed, a dual-encoder speech-text retrieval model trained on 73,476 speech-text pairs via InfoNCE contrastive learning, evaluated on the full candidate pool (~1,000 per language) and outperforming CLAP zero-shot across all eight languages and all metrics. Probing the fine-tuned encoder with t-SNE reveals language-family-aligned clustering tendencies without explicit family supervision, supported by a quantitative silhouette score (0.44 language-level, 0.11 family-level). All models, data splits, and utterance IDs are released under CC-BY-4.0; proprietary audio is freely available via an automated access request, used only for usage tracking.
Anthology ID:
2026.emnlp-main.2471
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
56109–56119
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2471/
DOI:
Bibkey:
Cite (ACL):
Badal Nyalang and Angana Borah. 2026. NE-MultiSpeech: A Multilingual Speech Corpus and ASR Benchmark for Northeast Indian Languages. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 56109–56119, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
NE-MultiSpeech: A Multilingual Speech Corpus and ASR Benchmark for Northeast Indian Languages (Nyalang & Borah, EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2471.pdf