Harshita Sharma
Author directory2026
Team CPS Lab at SHROOM-Visions: Parameter-Efficient Vision-Language Hallucination Detection with Threshold Tuning
Anika Srivastava | Vishal Kumar | Harshita Sharma | Nitisha Aggarwal | Geetika Jain Saxena | Amit Pundir | Sanjeev Singh
The Proceedings of the Third Workshop on Uncertainty-Aware NLP
Anika Srivastava | Vishal Kumar | Harshita Sharma | Nitisha Aggarwal | Geetika Jain Saxena | Amit Pundir | Sanjeev Singh
The Proceedings of the Third Workshop on Uncertainty-Aware NLP
Fine-grained large-vision-language hallucination detection presents the challenge of locating hallucinated content in image-grounded responses and classifying them with corresponding hallucination categories. Our method for the SHROOM-Visions 2026 Shared Task explores a parameter-efficient solution by fine-tuning a lightweight token-level classification head on the frozen weights of the Qwen3-VL-8B-Instruct vision-language model. We utilized category-specific thresholds calculated based on validation character-level IoU and designed a span merging post-processing step to facilitate span localization with fewer fragmented predictions. We also establish a possible connection between the annotator agreement and the operating thresholds. Our analysis suggests that, within our experimental setup, hallucination categories that had higher inter-annotator agreement tended to have higher optimal decision thresholds whereas categories with lower agreement tended to have lower optimal decision thresholds. Our parameter-efficient method achieved ranks 18th, 16th, 16th, and 15th place on the English, French, Italian, and Chinese tracks, respectively, showing promising performance for multilingual fine-grained hallucination detection.
CPS Lab 1@MMLow 2026: Hierarchical Multimodal Classification of Tamil Political Memes via Logit-Conditioned Stance-Target Prediction
Vishal Kumar | Anika Srivastava | Harshita Sharma | Nitisha Aggarwal | Geetika Jain Saxena | Amit Pundir | Sanjeev Singh
Proceedings of the Fifth Workshop on Multimodal Machine Learning in Low-resource Languages
Vishal Kumar | Anika Srivastava | Harshita Sharma | Nitisha Aggarwal | Geetika Jain Saxena | Amit Pundir | Sanjeev Singh
Proceedings of the Fifth Workshop on Multimodal Machine Learning in Low-resource Languages
Political memes have become an effective tool for online political communication and engagement. Their interpretation is particularly challenging in low-resource and code-mixed languages such as Tamil, where memes often include Tamil-English, use culturally relevant lexicon, incorporate political entities, and utilize references to prominent personalities and parties. This paper presents our submission to the MMLow shared task at AACL-IJCNLP 2026 on multi-level classification of Tamil political memes, which requires jointly predicting (i) a coarse-grained stance label (Support vs. Troll) and (ii) a fine-grained target label identifying whether the meme concerns a political party or a person. We proposed a lightweight hierarchical classifier built on frozen, pretrained multimodal encoders: images are represented using CLIP (ViT-B/32) and text using the multilingual BGE-M3 embedding model, with the two modalities combined via early, non-parametric concatenation fusion. A shared encoder feeds two prediction heads, where the target-identification head is explicitly conditioned on the stance head’s output logits via a gradient-detachment mechanism, allowing the model to respect the task’s label hierarchy without corrupting the coarse-level classifier’s gradients. To address pronounced class imbalance in both label levels, we applied balanced loss weighting at each level. On the test set of unlabeled data, our system achieved 3rd place, with macro-F1 scores of 0.856 for stance classification and 0.532 for target identification, yielding a final score of 0.694, indicating promising performance on the Tamil political meme classification problem.
CPSLab2@MMLow 2026: A Hierarchical Multimodal Framework with Stratified K-Fold Training for Political Meme Classification
Anika Srivastava | Vishal Kumar | Harshita Sharma | Nitisha Aggarwal | Geetika Jain Saxena | Amit Pundir | Sanjeev Singh
Proceedings of the Fifth Workshop on Multimodal Machine Learning in Low-resource Languages
Anika Srivastava | Vishal Kumar | Harshita Sharma | Nitisha Aggarwal | Geetika Jain Saxena | Amit Pundir | Sanjeev Singh
Proceedings of the Fifth Workshop on Multimodal Machine Learning in Low-resource Languages
Political memes communicate opinions using the interplay between visual and textual information, posing a challenging multimodal classification task. We present a hierarchical multimodal framework for political meme classification, which decomposes the task into stance and target entity prediction. Our framework was evaluated on the data from the Multi-Level Political Meme Classification Shared Task at MMLow @ AACL-IJCNLP 2026. The task comprised two levels of classification: (i) predicting the political stance of a meme as either Support/Praise or Troll/Oppose, and (ii) identifying the target of the meme combined with the stance. We approached the task by first predicting the Level-1 labels followed by predicting the target entity (Person/Party) and then combining them in a deterministic manner to predict the final Level-2 labels. Our method used a SigLIP vision transformer for stance classification and a hybrid SigLIP-XLM-RoBERTa architecture for target classification with late fusion. We trained our models using 5-fold stratified cross-validation with checkpoint ensembling. We also performed ablation experiments to analyze the contribution of individual modalities, observing that visual features are more informative for stance prediction while textual features contain more information for target identification, which supports the multimodal task at hand. Our system achieved a macro F1 score of 0.9027 and 0.6694 for Level-1 and Level-2 classification, respectively, achieving a final score of 0.7860, ranking first in the shared task.
2025
teamiic@DravidianLangTech2025-NAACL 2025: Transformer-Based Multimodal Feature Fusion for Misogynistic Meme Detection in Low-Resource Dravidian Language
Harshita Sharma | Simran | Vajratiya Vajrobol | Nitisha Aggarwal
Proceedings of the Fifth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages
Harshita Sharma | Simran | Vajratiya Vajrobol | Nitisha Aggarwal
Proceedings of the Fifth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages
Misogyny has become a pervasive issue in digital spaces. Misleading gender stereotypes are getting communicated through digital content.This content is majorly displayed as a text-and-image memes. With the growing prevalence of online content, it is essential to develop automated systems capable of detecting such harmful content to ensure safer online environments. This study focuses on the detection of misogynistic memes in two Dravidian languages, Tamil and Malayalam. The proposed model utilizes a pre-trained XLM-RoBERTa (XLM-R) model for text analysis and a Vision Transformer (ViT) for image feature extraction. A custom neural network classifier was trained on integrating the outputs of both modalities to form a unified representation. This model predicts whether the meme represents misogyny or not. This follows an early-fusion strategy since features of both modalities are combined before feeding into the classification model. This approach achieved promising results using a macro F1-score of 0.84066 on the Malayalam test dataset and 0.68830 on the Tamil test dataset. In addition, it is worth noting that this approach secured Rank 7 and 11 in Malayalam and Tamil classification respectively in the shared task of Misogyny Meme Detection (MMD). The findings demonstrate that the multimodal approach significantly enhances the accuracy of detecting misogynistic content compared to text-only or image-only models.
2024
MAIRA at RRG24: A specialised large multimodal model for radiology report generation
Shaury Srivastav | Mercy Ranjit | Fernando Pérez-García | Kenza Bouzid | Shruthi Bannur | Daniel C. Castro | Anton Schwaighofer | Harshita Sharma | Maximilian Ilse | Valentina Salvatelli | Sam Bond-Taylor | Fabian Falck | Anja Thieme | Hannah Richardson | Matthew P. Lungren | Stephanie L. Hyland | Javier Alvarez-Valle
Proceedings of the 23rd Workshop on Biomedical Natural Language Processing
Shaury Srivastav | Mercy Ranjit | Fernando Pérez-García | Kenza Bouzid | Shruthi Bannur | Daniel C. Castro | Anton Schwaighofer | Harshita Sharma | Maximilian Ilse | Valentina Salvatelli | Sam Bond-Taylor | Fabian Falck | Anja Thieme | Hannah Richardson | Matthew P. Lungren | Stephanie L. Hyland | Javier Alvarez-Valle
Proceedings of the 23rd Workshop on Biomedical Natural Language Processing
This paper discusses the participation of the MSR MAIRA team in the Large-Scale Radiology Report Generation Shared Task Challenge, as part of the BioNLP workshop at ACL 2024. We present a radiology-specific multimodal model designed to generate radiological reports from chest X-Rays (CXRs). Our proposed model combines a CXR-specific image encoder RAD-DINO with a Large Language Model (LLM) based on Vicuna-7B, via a multi-layer perceptron (MLP) adapter. Both the adapter and the LLM have been fine-tuned in a single-stage training setup to generate radiology reports. Experimental results indicate that a joint training setup with findings and impression sections improves findings prediction. Additionally, incorporating lateral images alongside frontal images when available further enhances all metrics. More information and resources about MAIRA can be found on the project website: http://aka.ms/maira.
2023
Verb Categorisation for Hindi Word Problem Solving
Harshita Sharma | Pruthwik Mishra | Dipti M. Sharma
Proceedings of the 20th International Conference on Natural Language Processing (ICON)
Harshita Sharma | Pruthwik Mishra | Dipti M. Sharma
Proceedings of the 20th International Conference on Natural Language Processing (ICON)
Word problem Solving is a challenging NLP task that deals with solving mathematical probglems described in natural language. Recently, there has been renewed interest in developing word problem solvers for Indian languages. As part of this paper, we have built a Hindi arithmetic word problem solver which makes use of verbs. Additionally, we have created verb categorization data for Hindi. Verbs are very important for solving word problems with addition/subtraction operations as they help us identify the set of operations required to solve the word problems. We propose a rule-based solver that uses verb categorisation to identify operations in a word problem and generate answers for it. To perform verb categorisation, we explore several approaches and present a comparative study.
Exploring the Boundaries of GPT-4 in Radiology
Qianchu Liu | Stephanie Hyland | Shruthi Bannur | Kenza Bouzid | Daniel Castro | Maria Wetscherek | Robert Tinn | Harshita Sharma | Fernando Pérez-García | Anton Schwaighofer | Pranav Rajpurkar | Sameer Khanna | Hoifung Poon | Naoto Usuyama | Anja Thieme | Aditya Nori | Matthew Lungren | Ozan Oktay | Javier Alvarez-Valle
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Qianchu Liu | Stephanie Hyland | Shruthi Bannur | Kenza Bouzid | Daniel Castro | Maria Wetscherek | Robert Tinn | Harshita Sharma | Fernando Pérez-García | Anton Schwaighofer | Pranav Rajpurkar | Sameer Khanna | Hoifung Poon | Naoto Usuyama | Anja Thieme | Aditya Nori | Matthew Lungren | Ozan Oktay | Javier Alvarez-Valle
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
The recent success of general-domain large language models (LLMs) has significantly changed the natural language processing paradigm towards a unified foundation model across domains and applications. In this paper, we focus on assessing the performance of GPT-4, the most capable LLM so far, on the text-based applications for radiology reports, comparing against state-of-the-art (SOTA) radiology-specific models. Exploring various prompting strategies, we evaluated GPT-4 on a diverse range of common radiology tasks and we found GPT-4 either outperforms or is on par with current SOTA radiology models. With zero-shot prompting, GPT-4 already obtains substantial gains (≈ 10% absolute improvement) over radiology models in temporal sentence similarity classification (accuracy) and natural language inference (F1). For tasks that require learning dataset-specific style or schema (e.g. findings summarisation), GPT-4 improves with example-based prompting and matches supervised SOTA. Our extensive error analysis with a board-certified radiologist shows GPT-4 has a sufficient level of radiology knowledge with only occasional errors in complex context that require nuanced domain knowledge. For findings summarisation, GPT-4 outputs are found to be overall comparable with existing manually-written impressions.
2022
HAWP: a Dataset for Hindi Arithmetic Word Problem Solving
Harshita Sharma | Pruthwik Mishra | Dipti Sharma
Proceedings of the Thirteenth Language Resources and Evaluation Conference
Harshita Sharma | Pruthwik Mishra | Dipti Sharma
Proceedings of the Thirteenth Language Resources and Evaluation Conference
Word Problem Solving remains a challenging and interesting task in NLP. A lot of research has been carried out to solve different genres of word problems with various complexity levels in recent years. However, most of the publicly available datasets and work has been carried out for English. Recently there has been a surge in this area of word problem solving in Chinese with the creation of large benchmark datastes. Apart from these two languages, labeled benchmark datasets for low resource languages are very scarce. This is the first attempt to address this issue for any Indian Language, especially Hindi. In this paper, we present HAWP (Hindi Arithmetic Word Problems), a dataset consisting of 2336 arithmetic word problems in Hindi. We also developed baseline systems for solving these word problems. We also propose a new evaluation technique for word problem solvers taking equation equivalence into account.
Search
Fix author
Co-authors
- Nitisha Aggarwal 4
- Geetika Jain Saxena 3
- Vishal Kumar 3
- Amit Pundir 3
- Sanjeev Singh 3
- Anika Srivastava 3
- Javier Alvarez-Valle 2
- Shruthi Bannur 2
- Kenza Bouzid 2
- Pruthwik Mishra 2
- Fernando Pérez-García 2
- Anton Schwaighofer 2
- Dipti Misra Sharma 2
- Anja Thieme 2
- Sam Bond-Taylor 1
- Daniel Castro 1
- Daniel C. Castro 1
- Fabian Falck 1
- Stephanie Hyland 1
- Stephanie L. Hyland 1
- Maximilian Ilse 1
- Sameer Khanna 1
- Qianchu Liu 1
- Matthew Lungren 1
- Matthew P. Lungren 1
- Aditya Nori 1
- Ozan Oktay 1
- Hoifung Poon 1
- Pranav Rajpurkar 1
- Mercy Ranjit 1
- Hannah Richardson 1
- Valentina Salvatelli 1
- Simran 1
- Shaury Srivastav 1
- Robert Tinn 1
- Naoto Usuyama 1
- Vajratiya Vajrobol 1
- Maria Wetscherek 1