Min Hyuk Kim
Author directory2026
K-Prism: A Korean Diagnostic Benchmark for Multi-faceted Hallucination in Vision-Language Models
Min Hyuk Kim | Dong Hyok Shin | Jimi Lee | Jin Yea Jang
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
Min Hyuk Kim | Dong Hyok Shin | Jimi Lee | Jin Yea Jang
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
Vision-language models (VLMs) often produce hallucinated outputs that are not supported by visual or textual evidence. While existing benchmarks have advanced hallucination evaluation, they primarily target object-level or task-specific errors and often assess different failure types on separate samples. This design limits direct comparison of hallucination patterns under shared evidence, and most prior evaluations remain English-centric. We introduce K-Prism, a Korean diagnostic benchmark for multi-faceted hallucination analysis in VLMs. K-Prism contains 1,002 multiple-choice questions across text and image tracks, organized into shared-evidence topic groups. Within each group, questions are designed to probe three diagnostic dimensions: factual, reasoning, and misleading. K-Prism uses a controlled dual-mode protocol comparing intact inputs with visually perturbed inputs to examine hallucination under reduced visual evidence. Experiments with closed- and open-source VLMs show dimension-specific degradation rather than a uniform accuracy drop. These findings demonstrate that hallucination in Korean VLMs cannot be captured by a single aggregate score and requires fine-grained diagnostic evaluation across evidence conditions and failure dimensions. Our dataset is publicly available at https://huggingface.co/spaces/KETI-NLP/K-Prism_Benchmark.
2025
X-FLoRA: Cross-modal Federated Learning with Modality-expert LoRA for Medical VQA
Min Hyuk Kim | Changheon Kim | Seok Bong Yoo
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Min Hyuk Kim | Changheon Kim | Seok Bong Yoo
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Medical visual question answering (VQA) and federated learning (FL) have emerged as vital approaches for enabling privacy-preserving, collaborative learning across clinical institutions. However, both these approaches face significant challenges in cross-modal FL scenarios, where each client possesses unpaired images from only one modality. To address this limitation, we propose X-FLoRA, a cross-modal FL framework that uses modality-expert low-rank adaptation (LoRA) for medical VQA. Specifically, X-FLoRA enables the synthesis of images from one modality to another without requiring data sharing between clients. This is achieved by training a backward translation model within a federated asymmetric translation scheme that integrates clinical semantics from textual data. Additionally, X-FLoRA introduces modality-expert LoRA, which fine-tunes separate LoRA modules to strengthen modality-specific representations in the VQA task. The server aggregates the trained backward translation models and fine-tuned LoRA modules using discriminator quality scores and expert-aware weighting, which regulate the relative contributions from different clients. Experiments were conducted on VQA datasets encompassing different medical modalities, and the results demonstrate that X-FLoRA outperforms existing FL methods in terms of VQA performance.
Search