K-Prism: A Korean Diagnostic Benchmark for Multi-faceted Hallucination in Vision-Language Models

Min Hyuk Kim, Dong Hyok Shin, Jimi Lee, Jin Yea Jang


Abstract
Vision-language models (VLMs) often produce hallucinated outputs that are not supported by visual or textual evidence. While existing benchmarks have advanced hallucination evaluation, they primarily target object-level or task-specific errors and often assess different failure types on separate samples. This design limits direct comparison of hallucination patterns under shared evidence, and most prior evaluations remain English-centric. We introduce K-Prism, a Korean diagnostic benchmark for multi-faceted hallucination analysis in VLMs. K-Prism contains 1,002 multiple-choice questions across text and image tracks, organized into shared-evidence topic groups. Within each group, questions are designed to probe three diagnostic dimensions: factual, reasoning, and misleading. K-Prism uses a controlled dual-mode protocol comparing intact inputs with visually perturbed inputs to examine hallucination under reduced visual evidence. Experiments with closed- and open-source VLMs show dimension-specific degradation rather than a uniform accuracy drop. These findings demonstrate that hallucination in Korean VLMs cannot be captured by a single aggregate score and requires fine-grained diagnostic evaluation across evidence conditions and failure dimensions. Our dataset is publicly available at https://huggingface.co/spaces/KETI-NLP/K-Prism_Benchmark.
Anthology ID:
2026.aacl-short.41
Volume:
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venues:
AACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
632–644
Language:
URL:
https://aclanthology.org/2026.aacl-short.41/
DOI:
Bibkey:
Cite (ACL):
Min Hyuk Kim, Dong Hyok Shin, Jimi Lee, and Jin Yea Jang. 2026. K-Prism: A Korean Diagnostic Benchmark for Multi-faceted Hallucination in Vision-Language Models. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 632–644, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
K-Prism: A Korean Diagnostic Benchmark for Multi-faceted Hallucination in Vision-Language Models (Kim et al., AACL-IJCNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aacl-short.41.pdf