Irum Shehryar
Author directory2026
Irums at SHROOM-Visions 2026: When One Fix Doesn’t Fit All — Category-Aware Hallucination Span Detection
Irum Shehryar
The Proceedings of the Third Workshop on Uncertainty-Aware NLP
Irum Shehryar
The Proceedings of the Third Workshop on Uncertainty-Aware NLP
This paper describes our system for the English subset of the SHROOM-Visions 2026 shared task, which requires detecting hallucinated spans in vision-language model outputs across Chinese, English, French, and Italian, and labeling them into five categories. We used Claude Haiku 4.5 as a zero-shot detector, with post-processing relabeling and Sonnet 5-based second-opinion verification for select categories. Our final system achieved 0.3015 (Cor+Lbl), 0.3949 (Cor) and 0.3421 (IoU). Our error analysis found that missed detections, low annotator agreement, and unresolved overlap between the invention and mischaracterization categories account for the majority of remaining errors, with additional score loss when the model finds the right content but its flagged text is a different length than gold’s, or is labeled with the wrong category. Systematically testing and comparing individual approaches shows that none of the methods we tried is effective across all categories on its own; category-specific, targeted fixes improved performance more reliably than any single general-purpose mechanism we tested.