Ranjit Raut

Author directory

2026

Vision-Language Models (VLMs) are showing remarkable performance in connecting vision and language, closely intertwined. However, they are vulnerable to hallucination and alignment problems. To address these issues, this paper describes a simple system for the SHROOM-visions shared task on character-level hallucination detection in image-conditioned text. The system attaches two lightweight linear heads to a pretrained VLM backbone and fine-tunes them, together with low-rank adapters on the backbone. As a result, they predict a per-character hallucination probability and a per-character distribution over five hallucination classes. We train directly against soft targets as the released annotations are themselves soft (overlapping, disagreeing multi-annotator spans), rather than collapsing them into hard span labels and fit a post-hoc isotonic calibration map so that predicted probabilities track empirical annotator agreement, the quantity the task’s Confidence Calibration metric rewards. The system we submitted to the official shared-task evaluation built on Qwen2.5-VL-3B-Instruct contained two implementation issues that we identified only after the submission deadline; we report the official results as submitted, document the issues and their fixes, and separately report unofficial post-evaluation results obtained after correcting them and porting the pipeline to Qwen3.5-4B.[The code is available at <https://github.com/theranjitraut/SHROOM-Visions-2026>]