Vishal Kumar
Author directory2026
Team CPS Lab at SHROOM-Visions: Parameter-Efficient Vision-Language Hallucination Detection with Threshold Tuning
Anika Srivastava | Vishal Kumar | Harshita Sharma | Nitisha Aggarwal | Geetika Jain Saxena | Amit Pundir | Sanjeev Singh
The Proceedings of the Third Workshop on Uncertainty-Aware NLP
Anika Srivastava | Vishal Kumar | Harshita Sharma | Nitisha Aggarwal | Geetika Jain Saxena | Amit Pundir | Sanjeev Singh
The Proceedings of the Third Workshop on Uncertainty-Aware NLP
Fine-grained large-vision-language hallucination detection presents the challenge of locating hallucinated content in image-grounded responses and classifying them with corresponding hallucination categories. Our method for the SHROOM-Visions 2026 Shared Task explores a parameter-efficient solution by fine-tuning a lightweight token-level classification head on the frozen weights of the Qwen3-VL-8B-Instruct vision-language model. We utilized category-specific thresholds calculated based on validation character-level IoU and designed a span merging post-processing step to facilitate span localization with fewer fragmented predictions. We also establish a possible connection between the annotator agreement and the operating thresholds. Our analysis suggests that, within our experimental setup, hallucination categories that had higher inter-annotator agreement tended to have higher optimal decision thresholds whereas categories with lower agreement tended to have lower optimal decision thresholds. Our parameter-efficient method achieved ranks 18th, 16th, 16th, and 15th place on the English, French, Italian, and Chinese tracks, respectively, showing promising performance for multilingual fine-grained hallucination detection.
AquaCrisis: A Multilingual Benchmark for Classifying Water-Utility Public Communication
Vishal Kumar | Mourad Oussalah
Proceedings of the 3rd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2026)
Vishal Kumar | Mourad Oussalah
Proceedings of the 3rd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2026)
Water utilities communicate both routine service information and time-sensitive public-risk guidance, yet multilingual NLP benchmarks for institution-authored infrastructure communication remain limited. We introduce AquaCrisis, a corpus of 20,000 text-bearing Facebook posts from 12 European water utilities, with a 1,500-post manually annotated gold standard. We formulate Task A as classification of a post’s main water-utility topic and Task B as classification of its primary communicative function, together with a coarser grouped Task B.1. We compare rule-based and supervised classifiers with four instruction-tuned LLMs on native-language and English-translated inputs. Supervised approaches achieve the highest point estimates on the fine-grained tasks: translated TF-IDF with logistic regression performs best on Task A, while translated multilingual sentence embeddings perform best on Task B. GPT-4o-mini achieves the highest point estimate on grouped Task B.1. Paired bootstrap analysis shows that translation effects are model- and setting-dependent: gains are clearest for the lexical TF-IDF classifier and selected Gemma 4 and GPT-4.1-mini conditions, while multilingual sentence embeddings, GPT-4o-mini, and Qwen 3.5 are comparatively stable across input representations. AquaCrisis provides a benchmark for multilingual water-utility communication spanning routine, educational, operational, and incident-related messaging, supporting communication monitoring, operational triage, and cross-lingual evaluation.
EduCoder: An Open-Source Annotation System for Education Transcript Data
Saad Ashraf | Jim Malamut | Vishal Kumar | Guanzhong Pan | HyunJi Nam | Mei Tan | Lucía Langlois | Liliana Carolina Santos-Deonizio | Helen Spencer Higgins | Dorottya Demszky
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
Saad Ashraf | Jim Malamut | Vishal Kumar | Guanzhong Pan | HyunJi Nam | Mei Tan | Lucía Langlois | Liliana Carolina Santos-Deonizio | Helen Spencer Higgins | Dorottya Demszky
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
We present EduCoder, an open-source web platform designed for annotating classroom conversation transcripts. Existing annotation tools do not support the team-based workflows or access to instructional context that education discourse research requires. EduCoder addresses these gaps by combining transcript text, synchronized video, and instructional materials within a single workspace. The platform supports scoping annotation to specific portions of a lesson, coordinating work across annotation teams, and optionally integrating LLM-generated annotations with structured human–LLM comparison. EduCoder is freely accessible at https://edu-coder.com.
2023
Evaluation Metrics for Depth and Flow of Knowledge in Non-fiction Narrative Texts
Sachin Pawar | Girish Palshikar | Ankita Jain | Mahesh Singh | Mahesh Rangarajan | Aman Agarwal | Vishal Kumar | Karan Singh
Proceedings of the 5th Workshop on Narrative Understanding
Sachin Pawar | Girish Palshikar | Ankita Jain | Mahesh Singh | Mahesh Rangarajan | Aman Agarwal | Vishal Kumar | Karan Singh
Proceedings of the 5th Workshop on Narrative Understanding
In this paper, we describe the problem of automatically evaluating quality of knowledge expressed in a non-fiction narrative text. We focus on a specific type of documents where each document describes a certain technical problem and its solution. The goal is not only to evaluate the quality of knowledge in such a document, but also to automatically suggest possible improvements to the writer so that a better knowledge-rich document is produced. We propose new evaluation metrics to evaluate quality of knowledge contents as well as flow of different types of sentences. The suggestions for improvement are generated based on these metrics. The proposed metrics are completely unsupervised in nature and they are derived from a set of simple corpus statistics. We demonstrate the effectiveness of the proposed metrics as compared to other existing baseline metrics in our experiments.