Prabhjot Singh

Author directory

2026

NLI debiasing research evaluates interventions on single architectures and datasets, leaving a critical assumption untested: do improvements reflect method quality or model capacity? We run 44 controlled experiments: eleven debiasing strategies across two architecturally distinct models (ELECTRA-small, RoBERTa-base) and two corpora (SNLI, MNLI), under identical conditions with Wilson 95% confidence intervals throughout. On ELECTRA-small, which achieves 87.4% in-domain accuracy, debiasing fails completely: all ten debiasing interventions fall within the baseline confidence interval on HANS (CoV = 0.75%). On RoBERTa-base, methods show genuine sensitivity (CoV = 10.76%, approximately 14x higher), yet the median intervention trends toward degrading OOD accuracy on SNLI, short of conventional significance (Wilcoxon p = 0.065), and method rankings across corpora are anti-correlated for ELECTRA (Spearman ρ = -0.42) but stable for RoBERTa (ρ = 0.75). Contrastive augmentation swings 5.6 pp across corpora (+3.59 pp MNLI, -2.02 pp SNLI). The positive robustness-accuracy correlation on RoBERTa (r = 0.848) inverts the assumed tradeoff: methods that improve HANS also preserve in-domain accuracy, while aggressive debiasers collapse both. The ELECTRA failure is not a pipeline artefact; the same codebase produces real gains on RoBERTa. It is a capacity failure: debiasing presupposes representational capacity that small architectures lack, a precondition the literature neither states nor tests.
Current multilingual evaluations for Vision-Language Models (VLMs) assume a one-to-one mapping between language and orthography, overlooking billions of users of multi-script languages. We introduce PuMVR (Punjabi Multimodal Visual Reasoning), a benchmark of 1,000 strictly parallel image-text instances across Punjabi’s three active scripts: Gurmukhi, Shahmukhi, and Roman. Evaluating 10 state-of-the-art VLMs, we expose a substantial and systematic Script Gap. Models frequently solve visual tasks in one script while failing identical tasks in another, with accuracy deltas reaching 16%. Crucially, visual input boosts absolute performance uniformly yet does not close the orthographic gap. Furthermore, cross-script in-context transfer is highly brittle, exposing script-locked knowledge representation. Supported by McNemar tests across all script pairs, our findings demonstrate that current "multilingual" VLMs are not truly multi-script. We propose the Script Consistency Rate (SCR), which falls as low as 24.8% on our benchmark, as a mandatory metric for script-agnostic evaluation to ensure equitable AI access.

2025

The widespread use of social media has contributed to the increase in hate speech and offensive language, impacting people of all ages. This issue is particularly difficult to address when the text is in a code-mixed language. Twitter is commonly used to express opinions in code-mixed language. In this paper, we introduce a novel Multi-Task Transfer Learning (MTTL) framework to detect aggression and offensive language. By focusing on the dual facets of cyberbullying, aggressiveness and offensiveness, our model leverages the MTTL approach to enhance the performance of the model on the aggression and offensive language detection. Results show that our Multi-Task Transfer Learning (MTTL) setup significantly enhances the performance of state-of-the-art pretrained language models, BERT, RoBERTa, and Hing-RoBERTa for Hindi-English code-mixed data from Twitter.