Workshop on Indian Language Data Resource and Evaluation (2026)
up
Proceedings of the 8th Workshop on Indian Language Data: Resources and Evaluation
Proceedings of the 8th Workshop on Indian Language Data: Resources and Evaluation
Girish Nath Jha | Kalika Bali | Sobha L | Devendr Kumar
Girish Nath Jha | Kalika Bali | Sobha L | Devendr Kumar
Many classical languages have well-studied traditions of poetic meter which enforce constraints on a poem in terms of syllable and phoneme patterns. Such advanced literary forms offer opportunities for probing deeper reasoning and language understanding in Large Language Models (LLMs) and their ability to follow strict pre-requisites and rules in generating text. In this paper, we introduce MetricalARGS, the first taxonomy of poetry-related NLP tasks designed to evaluate LLMs on metrical poetry across four dimensions: Analysis, Retrieval, Generation, and Support. We discuss how these tasks relate to existing NLP tasks, addressing questions around datasets and evaluation metrics. Taking the metrical poetry of Telugu language as our example, we illustrate how the taxonomy can be used with LLMs in practice through a quantitative and qualitative evaluation. MetricalARGS highlights the broader possibilities for understanding the capabilities and limitations of today’s LLMs through the lens of metrical poetry. We believe MetricalARGS can also serve as a reference taxonomy for studying and comparing metrical poetry across Indian languages as a starting point, and can be extended to other languages with established metrical poetry traditions.
Semi-automatic Approach for Tamil Discourse Relation Annotation
Frances Yung | Enosh Peter Ponraj | Vera Demberg
Frances Yung | Enosh Peter Ponraj | Vera Demberg
Discourse relations (DRs) specify the logical relations between text spans and are essential for modeling extended discourse. Resources annotated with DRs can help train large language models (LLMs) to recognize and generate these relations more naturally. However, there is currently no open-source DR-annotated resource for Tamil. Annotation is particularly challenging because many Tamil discourse connectives are realized as morphologically complex suffixes rather than standalone tokens, often involving phonological alternations. In this work, we present a DR-annotated dataset for Tamil based on the PDTB framework. We adopt a semi-automatic pipeline: 1) projection of automatic English discourse annotations onto Tamil in a parallel corpus; 2) lexical normalization using a morphological analyzer; and 3) manual verification of each instance. The resulting resource contains approximately 7;200 explicit DR annotations and a lexicon of 450 Tamil discourse connectives. The annotated data is available for download at https://anonymous.4open.science/r/Tamil-Semi-Automatic-Discourse-Relation-Dataset/.
Konkani Daan: A Community-Driven Culturally Grounded Speech Corpus for Low-Resource ASR
Milind Shivolkar | Vaibhav Gawas | Jyoti Pawar
Milind Shivolkar | Vaibhav Gawas | Jyoti Pawar
Indian languages are deeply embedded in cultural traditions, oral narratives, regional lexicons, and socially grounded communicative practices. However, existing speech resources and large multilingual ASR models often underrepresent culturally rich and naturally occurring speech varieties. In this paper, we introduce Konkani Daan, a community-driven initiative for collecting culturally grounded speech data for the Konkani language. The corpus currently comprises over 43.9 hours of 16 kHz speech recordings contributed through a web-based participatory platform. We evaluate a strong multilingual baseline, AI4Bharat IndicConformer-600M, in zero-shot mode on the Konkani Daan development set (379 utterances), achieving Word Error Rate (WER) of 46.46% and Character Error Rate (CER) of 15.47%, indicating substantial domain and cultural mismatch. Through qualitative error analysis, we identify systematic challenges including compound word segmentation, numeric normalisation, named entity distortion, and orthographic variation. Our findings demonstrate that culturally dense community speech exposes systematic limitations in multilingual ASR systems and motivates normalisation-aware and culturally informed evaluation strategies.
Is Literal Annotation Enough? Building an Annotation Framework for Metonymic Named Entities in Marathi
Pratibha Dongare
Pratibha Dongare
Named Entity Recognition (NER) has been a core task of natural language processing (NLP) since the Message Understanding Conferences (MUCs). Data annotation plays a crucial role in this task. However, existing annotation studies often rely on the literal sense of entities. Such annotations may lead to inconsistencies, while resolving ambiguity introduced by figurative tropes like metonymy. For example, in India won the series, India refers to a sports team instead of a geographic location. Understanding such non-literal senses is crucial for various NLP applications such as Question Answering, Information Extraction, etc. By addressing this gap, this study presents an annotation framework and detailed guidelines for annotating metonymic readings of named entities in Marathi, an Indo-Aryan language spoken in the central-western region of India. The study uses news corpus from various domains. It presents a two-tiered annotation framework for annotating conventional metonymies in Marathi language. Further, it describes the annotation framework applied to a corpus of 1,279 Marathi sentences. The result shows the inadequacy of literal-only annotation as 53.6% of named entity spans have metonymic readings. This study makes a crucial contribution for resource development for low-resource languages that share similar linguistic structures and cultural contexts. The paper describes the framework with necessary examples, challenges and concludes with a future scope.
Bengali-English and Hindi-English Code Mixed Speech Data with Disfluencies
Anuran Mitra | Tapabrata Mondal | Anirvan Chakravarty | Sivaji Bandyopadhyay
Anuran Mitra | Tapabrata Mondal | Anirvan Chakravarty | Sivaji Bandyopadhyay
Spontaneous speech in multilingual communities such as India frequently combines code-switching (CS) and disfluencies, yet existing Bengali–English and Hindi–English speech corpora largely consist of fluent or scripted utterances. This limits their suitability for developing and evaluating automatic speech recognition (ASR) systems intended for real conversational settings, particularly in micro-resource scenarios. We introduce BEHE-CMDisfl, a synthetic speech corpus that explicitly integrates disfluency phenomena within Bengali–English and Hindi–English code-mixed (CM) utterances. The textual content was generated using prompting strategies with large language models (LLMs) to encourage controlled switching and varied disfluency patterns, including filled pauses, repetitions, and restarts. The utterances were subsequently synthesized using Indic Parler text-to-speech (TTS) system. To demonstrate usability, we establish a reproducible GMM–HMM baseline for Bengali–English ASR using Kaldi on a 1.3-hour subset of the corpus. In our experiments, improvements were mainly observed after ensuring consistency in the pronunciation lexicon and applying phonetic normalization, with the best setup reaching a word error rate (WER) of 37.74%. A closer look at the decoded transcripts suggests that filled pauses and repetitions are not automatically collapsed, but appear in the output, indicating that the disfluency cues present in the synthetic speech are captured during recognition.
Konkani is a low-resource Indo-Aryan language spoken along the western coast of India, characterized by significant dialectal variation, multi-script usage, and limited standardized computational resources. This paper presents a consolidated and analysis-ready lexical resource derived from the Konkani Wordnet, built under the IndoWordNet framework. The resource comprises 32,370 synsets, 37,719 unique lexical entries, 32,370 glosses, and 33,318 example sentences, enriched with pronunciations, semantic relations, and illustrative examples. We describe the systematic extraction, normalization, and structural integration of wordnet data, resolving identifier inconsistencies and ensuring semantic coherence across distributed lexical files. To demonstrate the practical utility of this resource, we present an API-based bilingual vocabulary exercise generation system that leverages shared synset identifiers to automatically produce semantically aligned Hindi–Konkani word pairs for e-learning applications. The resulting resource enhances accessibility, reproducibility, and computational readiness for NLP tasks, while providing a foundational infrastructure for developing technology-driven teaching and learning tools for Konkani.
Development of Speech Corpus for Low-Resource Language- a Case of Sanskrit
Devendr Kumar | Girish Nath Jha | Khalid Choukri
Devendr Kumar | Girish Nath Jha | Khalid Choukri
This paper presents a comprehensive framework for the development of a speech corpus for Sanskrit, designed to facilitate advances in Automatic Speech Recognition (ASR) and AI/ML research. The proposed corpus comprises over 107 hours of transcribed speech data, collected from diverse Sanskrit sources through a systematic and scalable pipeline. We detail the end-to-end methodology adopted for corpus creation, encompassing web crawling, data sanitization, audio downloading, and transcription alignment. Particular emphasis is placed on the methodological rigor applied at each stage, including source selection, preprocessing for quality assurance, transcription protocols, and forced alignment techniques. The paper further addresses the unique complexities inherent to Sanskrit, spanning its phonetic richness, intricate morphological structure, and distinctive syntactic patterns. By systematically addressing these dimensions, the resulting 107-hour corpus aims to serve as a foundational resource for speech technology research in Sanskrit.
The Shabd Portal – Searchable Lexical Resources for Indian Languages by Government of India
Mercy Lalrohluo Hmar | Girish Nath Jha | Dhananjay Singh
Mercy Lalrohluo Hmar | Girish Nath Jha | Dhananjay Singh
The shabd portal ( https://shabd.education.gov.in ) of Commission for Scientific and Technical Terminology (CSTT), a subordinate office under the Ministry of Education, Department of Higher Education, Government of India (GOI) is a data server designed and developed by Prof. Girish Nath Jha, former Chairman CSTT, featuring all the standardized scientific and technical glossaries of CSTT in digital searchable mode. The aim is to launch a central repository for the terminologies prepared in Indian Languages, thus enriching the language bank of India enabling user friendly and free access to standardized terminology. This website is available in 22 Indian languages. The data covers several domains of science, humanities, engineering, medical science and agriculture subjects. The data is dynamic with regular updates in various domains. Users can search the equivalents of terms in Indian languages and submit their feedback for those equivalents prepared by CSTT. The unique feature of the search platform is that users have various options for search, based on languages, subjects, dictionary type and language pairs. The user can also choose to search in a specific glossary or the entire collection which includes about 471 glossaries having about (29,56,125 headwords).
POS Tagging in Low-Resource Maithili Language: Specific Challenges and Nuances
Shivani Priya | Shruti Jha | Urmila Jha | Girish Nath Jha | Deepali Tiwari | Jyoti Raj
Shivani Priya | Shruti Jha | Urmila Jha | Girish Nath Jha | Deepali Tiwari | Jyoti Raj
Abstract Part-of-Speech (POS) tagging is a key step in Natural Language Processing (NLP), laying the groundwork for more advanced syntactic and semantic tasks. Despite Maithili’s status as an Indo-Aryan language with a rich literary tradition and official recognition in India, computational resources for it are still very limited. In this paper, the creation of an annotated corpus of 25,000 sentences drawn from the fields of health, tourism, and administration is described with the hierarchical tagset currently used for Maithili. This paper also indicates that standard tagsets, typically adapted from English or Hindi, fail to capture the linguistic nuances of Maithili. This underestimates the need for a dedicated tagging framework that considers characteristics like vocative particles, verbal nuances, honorific complexities. Keywords: Parts of Speech, Natural Language Processing, Maithili, annotation
Preserving Civilisation Memory: A Digital Humanities Approach to the Ramayan
Shashank Tiwari | Girish Nath Jha
Shashank Tiwari | Girish Nath Jha
Abstract The Ramayana takes a leading role in the list of the most important texts in the world literature, with a multiplicity of textual traditions of unparalleled numbers of more than three hundred variants throughout South and Southeast. Stone manuscripts, bamboo manuscripts, and palm leaf manuscripts have been passed down in palm-leaf codices, inscriptions on temple walls, in highly illustrated folios and through generations of oral performance. But the corpus now faces serious challenges due to the destruction of the environment, material frailty, fragmentation, and the scope of modern script recognition methods. The current analysis examines how digitization projects are re-defining the preservation of Ramayana in a heritage system that is networked across the world. In this paper, the author critically assesses the work of large-scale projects thru the use of a qualitative research design that has been conducted between the years 2003 and 2026, including the National Mission for Manuscripts (NMM), the digital reunification of the Mewar Ramayana, and efforts by southeast Asian countries to document adaptations, like the Reamker by Cambodia). It predicts imaging standards, metadata formatting policies, integration of optical character recognition (OCR) and digital access structures, and struggles with the problem of multi-script complexity (Grantha, Devanagari, Kawi), partial coverage of variant texts, and infrastructural inequities. The results support that digitization has a significant positive impact on scholarly accessibility and comparative research but the advantages are unexpressed, especially relating to oral traditions. To make the endeavor sustainable preservation, interoperable standards must be adopted, the script recognition with the help of AI should be encouraged, the community should be involved, and cross-border collaboration institutionalized to protect the long-term cultural viability of the Ramayana. Keywords: Ramayana, Manuscript Preservation, Digitization of Cultural Heritage, Digital Humanities, Textual Transmission, Palm-Leaf Manuscripts, Grantha and Kawi Scripts, Metadata Standards (METS/XML), IIIF Interoperability, AI-Assisted Philology, Intangible, Cultural Heritage, Archival Sustainability, Cultural Heritage Informatics, Open Access Repositories, Civilizational Memory.
Naamah: A Large Scale Synthetic Sanskrit NER Corpus via DBpedia Seeding and LLM Generation
Annarao Kulkarni | Akhil Rajeev P
Annarao Kulkarni | Akhil Rajeev P
The digitization of classical Sanskrit literature is impeded by a scarcity of annotated resources, particularly for Named Entity Recognition (NER). While recent methodologies utilize generic Large Language Models (LLMs) for data augmentation, these approaches remain prone to error and often lack the reasoning depth required for classical grammar. In this work, we introduce Naamah, a high quality silver standard Sanskrit NER dataset comprising 102,942 sentences. We propose a methodology that combines entity extraction from DBpedia with the generative capabilities of a 24B parameter hybrid reasoning model to create grammatically natural and synthetically diverse training data. We utilize this dataset to benchmark two transformer architectures: the massive multilingual XLM RoBERTa and the parameter efficient IndicBERTv2. Our experiments reveal a key insight: while both models scale well with synthetic data, IndicBERTv2 qualitatively outperforms XLM RoBERTa in entity identification and classification. On a fixed split of 92,647 train and 10,295 validation examples, IndicBERTv2 achieves the best validation F1 of 0.9615, outperforming XLM R’s 0.9506 while remaining substantially lighter for deployment. We demonstrate that the generic tokenizer of XLM R fractures Sanskrit terms, whereas the domain adapted tokenizer of IndicBERTv2 preserves semantic integrity.
IndEuph-170: Benchmarking Cultural Pragmatics through Euphemism Detection in Indian English
Debamita Samajdar
Debamita Samajdar
Large Language Models (LLMs) have shown remarkable proficiency in standard English benchmarks, yet their ability to navigate the sociopragmatic cues of non-Western English varieties remains underexplored. This paper introduces IndEuph-170, a novel benchmark dataset focused on Indian English (IndE) euphemisms — expressions whose roots lie in local social hierarchies, politeness norms, and cultural taboos (e.g., "setting," "loose character," "suitable boy"). IndEuph-170 comprises 170 curated IndE sentences, against which the performance of two distinct architectures was evaluated: a fine-tuned BART model and GPT-4. The findings reveal a significant "cultural gap". While GPT-4 achieves 82.5% accuracy, it struggles with authoritative and punitive nuances. BART achieves 55.3% accuracy but exhibits a high rate of false positives by over-classifying general Indianisms as euphemisms. The paper argues that current multilingual benchmarks such as MME (Fu et al., 2025) and GLUE (Wang et al., 2018) fail to capture these dialectal pragmatics, and that a culturally-aware evaluation framework for Global Englishes is necessary.
Integrating Syntactic and Discourse Signals through Multi-Encoder Fusion in NMT for Low-Resource Indian Language Pairs
Sobha Lalitha Devi | Vijay Sundar Ram | Pattabhi RK Rao
Sobha Lalitha Devi | Vijay Sundar Ram | Pattabhi RK Rao
Neural Machine Translation (NMT) for low-resource Indian language pairs such as Hindi–Tamil and Tamil–Malayalam remains challenging due to morphological richness, syntactic divergence, and limited availability of high-quality parallel corpora. While Transformer-based architectures achieve strong performance in high-resource settings, they often struggle to model syntactic structure and discourse-level dependencies in low-resource scenarios, resulting in errors in agreement, word order, and pronoun translation. In this work, we propose a linguistically informed multi-encoder fusion framework that explicitly incorporates syntactic and discourse signals into NMT. Experiments conducted on Hindi–Tamil and Tamil–Malayalam parallel corpora demonstrate consistent improvements over strong Transformer baselines in BLEU and ChrF scores, along with gains in pronoun translation accuracy and agreement consistency. The results highlight the effectiveness of explicit linguistic integration for improving NMT in low-resource Indian language settings.
NE-LID: A Fast and Accurate Language Identification System for Northeast Indian Languages
Badal Nyalang
Badal Nyalang
Language identification (LID) is crucial for natural language processing systems, yet Northeast Indian languages remain severely underserved by existing multilingual LID models. We present NE-LID, a fast and accurate language identification system specifically designed for eleven languages of Northeast India. Built using character n-gram features with fastText, NE-LID achieves 99.09% accuracy on a balanced test set, significantly outperforming existing multilingual systems including GlotLID (73.12%), OpenLID (42.03%), IndicLID (39.30%), and LangDetect (24.33%). Our model processes predictions in 0.084 milliseconds on average, enabling real-time applications. We demonstrate that character-level modeling outperforms transformer-based approaches for script-diverse, low-resource languages
Integrating Cultural Wisdom and Digital Technologies for Children’s Moral and Emotional Development
Ms Garima | Girish Nath Jha
Ms Garima | Girish Nath Jha
The influence of technology in children’s education is increasing rapidly in the digital age, but with it the challenge of how to develop the cultural and moral development of children in a balanced manner in the technological environment. In traditional societies, moral and cultural teachings have often been imparted through religious and philosophical texts, memorization, interpretation, and oral tradition. This research presents an AI-based value-based learning framework, which aims to make cultural and ethical teachings more structured, simple, and technologically accessible to children. The study provides a brief analysis of memory-based teaching systems prevalent in various religious traditions and presents a model based on verses from the Bhagavad Gita as an example. The proposed system includes data generation and processing, simplified interpretation, semantic understanding, pronunciation analysis and interactive learning facilities based on selected material from cultural texts. The study indicates that through AI and modern technologies, traditional cultural knowledge can be delivered to children in a more effective and engaging form, developing new possibilities for reinforcing their moral and cultural development.