Irum Shehryar

Author directory

2026

This paper describes our system for the English subset of the SHROOM-Visions 2026 shared task, which requires detecting hallucinated spans in vision-language model outputs across Chinese, English, French, and Italian, and labeling them into five categories. We used Claude Haiku 4.5 as a zero-shot detector, with post-processing relabeling and Sonnet 5-based second-opinion verification for select categories. Our final system achieved 0.3015 (Cor+Lbl), 0.3949 (Cor) and 0.3421 (IoU). Our error analysis found that missed detections, low annotator agreement, and unresolved overlap between the invention and mischaracterization categories account for the majority of remaining errors, with additional score loss when the model finds the right content but its flagged text is a different length than gold’s, or is labeled with the wrong category. Systematically testing and comparing individual approaches shows that none of the methods we tried is effective across all categories on its own; category-specific, targeted fixes improved performance more reliably than any single general-purpose mechanism we tested.