Dong Hyok Shin
Author directory2026
K-Prism: A Korean Diagnostic Benchmark for Multi-faceted Hallucination in Vision-Language Models
Min Hyuk Kim | Dong Hyok Shin | Jimi Lee | Jin Yea Jang
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
Min Hyuk Kim | Dong Hyok Shin | Jimi Lee | Jin Yea Jang
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
Vision-language models (VLMs) often produce hallucinated outputs that are not supported by visual or textual evidence. While existing benchmarks have advanced hallucination evaluation, they primarily target object-level or task-specific errors and often assess different failure types on separate samples. This design limits direct comparison of hallucination patterns under shared evidence, and most prior evaluations remain English-centric. We introduce K-Prism, a Korean diagnostic benchmark for multi-faceted hallucination analysis in VLMs. K-Prism contains 1,002 multiple-choice questions across text and image tracks, organized into shared-evidence topic groups. Within each group, questions are designed to probe three diagnostic dimensions: factual, reasoning, and misleading. K-Prism uses a controlled dual-mode protocol comparing intact inputs with visually perturbed inputs to examine hallucination under reduced visual evidence. Experiments with closed- and open-source VLMs show dimension-specific degradation rather than a uniform accuracy drop. These findings demonstrate that hallucination in Korean VLMs cannot be captured by a single aggregate score and requires fine-grained diagnostic evaluation across evidence conditions and failure dimensions. Our dataset is publicly available at https://huggingface.co/spaces/KETI-NLP/K-Prism_Benchmark.
KoMMERCE: A Korean Multimodal and Multitask Evaluation Benchmark for Reliable Conversational E-commerce Agents
Byunggill Joe | San Kim | Minyoung Jung | Dong Hyok Shin | Yu-Jeong Sung | Saim Shin
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Byunggill Joe | San Kim | Minyoung Jung | Dong Hyok Shin | Yu-Jeong Sung | Saim Shin
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
E-commerce conversations are increasingly multimodal, yet end-to-end response scores cannot localize which dialogue competence failed, and no public benchmark evaluates finegrained multimodal dialogue understanding for Korean commercial agents. We present KoMMERCE, a Korean multimodal fashion-domain benchmark that coordinates nine fine-grained dialogue understanding tasks on the same conversations: sentiment, intent, and strategy classification;image and text knowledge retrieval;dialogue state tracking; span detection; coreference resolution; and the first formulation of cross-modal dialogue relation extraction. The corpus contains 15,293 dyadic role-play dialogues under two-stage quality control. Baselines on six frontier multimodal LLMs reveal task-conditional patterns that should inform deployment choices: no model wins all nine tasks, and context length affects accuracy differently from one task to another.