Milind Shivolkar
2026
Konkani Daan: A Community-Driven Culturally Grounded Speech Corpus for Low-Resource ASR
Milind Shivolkar | Vaibhav Gawas | Jyoti Pawar
Proceedings of the 8th Workshop on Indian Language Data: Resources and Evaluation
Milind Shivolkar | Vaibhav Gawas | Jyoti Pawar
Proceedings of the 8th Workshop on Indian Language Data: Resources and Evaluation
Indian languages are deeply embedded in cultural traditions, oral narratives, regional lexicons, and socially grounded communicative practices. However, existing speech resources and large multilingual ASR models often underrepresent culturally rich and naturally occurring speech varieties. In this paper, we introduce Konkani Daan, a community-driven initiative for collecting culturally grounded speech data for the Konkani language. The corpus currently comprises over 43.9 hours of 16 kHz speech recordings contributed through a web-based participatory platform. We evaluate a strong multilingual baseline, AI4Bharat IndicConformer-600M, in zero-shot mode on the Konkani Daan development set (379 utterances), achieving Word Error Rate (WER) of 46.46% and Character Error Rate (CER) of 15.47%, indicating substantial domain and cultural mismatch. Through qualitative error analysis, we identify systematic challenges including compound word segmentation, numeric normalisation, named entity distortion, and orthographic variation. Our findings demonstrate that culturally dense community speech exposes systematic limitations in multilingual ASR systems and motivates normalisation-aware and culturally informed evaluation strategies.
Unigoa@CHiPSAL 2026: Early vs Late Fusion for Multimodal Hate and Sentiment Detection in Nepali Memes
Ashweta Fondekar | Milind Shivolkar | Jyoti Pawar
Proceedings of the Second workshop on Challenges in Processing South Asian Languages (CHiPSAL2026)
Ashweta Fondekar | Milind Shivolkar | Jyoti Pawar
Proceedings of the Second workshop on Challenges in Processing South Asian Languages (CHiPSAL2026)
Internet memes pose significant challenges for automatic content moderation due to the interaction of visual and textual cues, sarcasm, and cultural context. In this work, we participate in the CHiPSAL 2026 shared task on multimodal hate and sentiment understanding in Nepali memes. The task consists of two subtasks: binary hate speech detection and three-class sentiment classification. We investigate both early-fusion and late-fusion multimodal architectures. Our primary system employs a late-fusion dual-encoder architecture combining XLM-RoBERTa for multilingual text representation and CLIP for visual encoding. We further evaluate an early-fusion ViLT-based joint vision–language transformer using NepBERTa tokenization as a baseline. Experimental results show that late-fusion models consistently outperform early-fusion architectures, particularly for code-mixed memes containing Devanagari Nepali and Roman-script English text. Our best system achieves a Macro-F1 of 0.6564 for hate speech detection and 0.4859 for sentiment classification. We provide analysis highlighting the challenges of multilingual code-mixing, sarcasm, and implicit sentiment in low-resource multimodal settings.