Shubham Sharma

Author directory

2026

This paper describes our system for SemEval 2026 Task 10 (PsyCoMark), focusing on Subtask 2: binary conspiracy classification in Reddit submission statements. We present a heterogeneous ensemble approach that combines Transformer-based models (DeBERTa, RoBERTa) with State-Space Models (Mamba) to leverage architectural diversity for improved generalization. Our key contributions include: (1) Bidirectional Mamba (BiMamba), adapting state-space sequence models for bidirectional document classification; (2) (2) a safety-switched multi-task training setup that uses marker supervision only for gold-annotated samples, preventing noisy pseudo-labeled rows from affecting the span extraction objective; and (3) Confidence-Based Iterative Refinement, using committee voting for high-quality pseudo-label generation. Our best official submission achieved a weighted F1 score of 0.78 on the Subtask 2 test set, ranking 4th on the public CodaBench leaderboard. We provide detailed ablation studies demonstrating the complementary contributions of each architectural component to inform future research directions.
Target-speaker extraction (TSE) systems are typically evaluated only after the complete output has been produced, limiting the ability to identify potentially unreliable outputs during generation. We investigate whether failure risk can instead be estimated causally from only the first three seconds of the emitted TSE waveform. We develop an early-warning monitor using 11,946 outputs spanning five conversational corpora and six TSE systems. Because independent failure labels are unavailable for this discovery cohort, we define a proxy adverse outcome from seven complete-output indicators of degraded or atypical TSE behaviour. After excluding monitoring features that overlap with the proxy construction, the monitor predicts this eventual outcome from the early waveform with an AUROC of 0.8062 and an AUPRC of 0.6606. At a 15% review budget, it identifies 54.47% of proxy-adverse outputs with 66.29% precision. To test whether this early-warning signal transfers beyond the discovery setting, we freeze the monitor and evaluate it unchanged on 2,000 unseen outputs with independently computed SI-SDRi. When failure is defined as SI-SDRi ≤ 0 dB, performance falls to near chance (AUROC 0.5354; AUPRC 0.0700). Yet, on the same external outputs, the frozen predictions remain strongly associated with the original proxy outcome (AUROC 0.8092; AUPRC 0.8481). The proxy and SI-SDRi failure definitions agree only weakly ( κ = 0 .028, MCC= 0 .064), while substantial featuredistribution shift is observed between the discovery and external cohorts (45 of 63 features have PSI ≥ 0.25). These results show that short-prefix monitoring can provide useful early risk prioritization within the setting in which the monitor is developed, but that such signals do not automatically transfer to independently defined failure outcomes. Reliable deployment therefore requires explicit attention to both distribution shift and the definition of failure.

2025

As digital platforms redefine educational paradigms, ensuring interactivity remains vital for effective learning. This paper explores using Multimodal Large Language Models (MLLMs) to automatically respond to student questions from online lectures - a novel question answering task of real world significance. We introduce the EduVidQA Dataset with 5252 question-answer pairs (both synthetic and real-world) from 296 computer science videos covering diverse topics and difficulty levels. To understand the needs of the dataset and task evaluation, we empirically study the qualitative preferences of students, which we provide as an important contribution to this line of work. Our benchmarking experiments consist of 6 state-of-the-art MLLMs, through which we study the effectiveness of our synthetic data for finetuning, as well as showing the challenging nature of the task. We evaluate the models using both text-based and qualitative metrics, thus showing a nuanced perspective of the models’ performance, which is paramount to future work. This work not only sets a benchmark for this important problem, but also opens exciting avenues for future research in the field of Natural Language Processing for Education.

2024

This paper describes our submission to the Cross-Lingual Classification of Corporate Social Responsibility (CSR) Themes and Topics shared task, aiming to identify themes and fine-grained topics present in news articles. Classifying news articles poses several challenges, including limited training data, noisy articles, and longer context length. In this paper, we explore the potential of using pretrained transformer models to classify news articles into CSR themes and fine-grained topics. We propose two different approaches for these tasks. For multi-class classification of CSR themes, we suggest using a pretrained multi-lingual encoder-based model like microsoft/mDeBERTa-v3-base, along with a variable selection network to classify the article into CSR themes. To identify all fine-grained topics in each article, we propose using a pretrained encoder-based model like Longformer, which offers a higher context length. We employ chunking-based inference to avoid information loss in inference and experimented with using different parts and manifestation of original article for training and inference.

2016

We propose a classification framework for semantic type identification of compounds in Sanskrit. We broadly classify the compounds into four different classes namely, Avyayībhāva, Tatpuruṣa, Bahuvrīhi and Dvandva. Our classification is based on the traditional classification system followed by the ancient grammar treatise Adṣṭādhyāyī, proposed by Pāṇini 25 centuries back. We construct an elaborate features space for our system by combining conditional rules from the grammar Adṣṭādhyāyī, semantic relations between the compound components from a lexical database Amarakoṣa and linguistic structures from the data using Adaptor Grammars. Our in-depth analysis of the feature space highlight inadequacy of Adṣṭādhyāyī, a generative grammar, in classifying the data samples. Our experimental results validate the effectiveness of using lexical databases as suggested by Kulkarni and Kumar (2013) and put forward a new research direction by introducing linguistic patterns obtained from Adaptor grammars for effective identification of compound type. We utilise an ensemble based approach, specifically designed for handling skewed datasets and we achieve an overall accuracy of 0.77 using random forest classifiers.