Sonal Khosla

Author directory

2026

This paper investigates the near-total absence of Urdu from mainstream LLM safety evaluation and nine years of WOAH proceedings. A complete enumeration of all 205 papers across nine ALW/WOAH editions confirms zero dedicated Urdu papers across the entire period. To assess whether this absence has measurable consequences, five large language models (GPT-4o, Claude Sonnet, Gemini 2.5 Flash, Llama-3.1, and Qwen-2.5) were evaluated across six datasets spanning Nastaliq Urdu, Roman Urdu, English, and code-switched Urdu-English, producing approximately 28,000 classifications. Label instability between original-script and English-translation classification ranged from 15.9% to 31.6%, and a missed-in-Urdu rate content flagged as harmful in English but passed as normal in Urdu ranged from 2.4% to 9.9%. Pairwise significance testing confirms that frontier closed models form a statistically indistinguishable cluster significantly outperforming smaller open-weight models on both metrics. The paper also identifies the absence of a dedicated Nastaliq Urdu with English code-switched hate speech dataset as a structural gap, and raises questions about annotation bias in existing Urdu corpora. The findings demonstrate that Urdu speakers are exposed to measurably weaker content moderation safety assurance, a gap that this paper documents, quantifies, and calls on the research community to address, considering that it is in the top 10 most spoken and used language in the world.

2025

Quantization is an essential and popular technique for improving the accessibility of large language models (LLMs) by reducing memory usage and computational costs while maintaining performance. In this study, we apply 4-bit Group Scaling Quantization (GSQ) and Generative Pretrained Transformer Quantization (GPTQ) to LLaMA 1B, Qwen 0.5B, and PHI 1.5B, evaluating their impact across multiple NLP tasks. We benchmark these models on MS MARCO (Information Retrieval), BoolQ (Boolean Question Answering), and GSM8K (Mathematical Reasoning) datasets, assessing both accuracy and efficiency accross various tasks. The study measures the trade-offs between model compression and task performance, analyzing key evaluation metrics namely: accuracy, inference latency, and throughput, providing insights into the suitability of low-bit quantization for real-world deployment and highlight the tradeoffs between memory, computing and latency in such settings, helping a user make suitable decisions
This paper introduces OVQA, the first multimodal dataset designed for visual question-answering (VQA), visual question elicitation (VQE), and multimodal research for the low-resource Odia language. The dataset was created by manually translating 6,149 English question-answer pairs, each associated with 6,149 unique images from the Visual Genome dataset. This effort resulted in 27,809 English-Odia parallel sentences, ensuring a semantic match with the corresponding visual information. Several baseline experiments were conducted on the dataset, including visual question answering and visual question elicitation. The dataset is the first VQA dataset for the low-resource Odia language and will be released for multimodal research purposes and also help researchers extend for other low-resource languages.