Min Hyuk Kim

Author directory

2026

Vision-language models (VLMs) often produce hallucinated outputs that are not supported by visual or textual evidence. While existing benchmarks have advanced hallucination evaluation, they primarily target object-level or task-specific errors and often assess different failure types on separate samples. This design limits direct comparison of hallucination patterns under shared evidence, and most prior evaluations remain English-centric. We introduce K-Prism, a Korean diagnostic benchmark for multi-faceted hallucination analysis in VLMs. K-Prism contains 1,002 multiple-choice questions across text and image tracks, organized into shared-evidence topic groups. Within each group, questions are designed to probe three diagnostic dimensions: factual, reasoning, and misleading. K-Prism uses a controlled dual-mode protocol comparing intact inputs with visually perturbed inputs to examine hallucination under reduced visual evidence. Experiments with closed- and open-source VLMs show dimension-specific degradation rather than a uniform accuracy drop. These findings demonstrate that hallucination in Korean VLMs cannot be captured by a single aggregate score and requires fine-grained diagnostic evaluation across evidence conditions and failure dimensions. Our dataset is publicly available at https://huggingface.co/spaces/KETI-NLP/K-Prism_Benchmark.

2025

Medical visual question answering (VQA) and federated learning (FL) have emerged as vital approaches for enabling privacy-preserving, collaborative learning across clinical institutions. However, both these approaches face significant challenges in cross-modal FL scenarios, where each client possesses unpaired images from only one modality. To address this limitation, we propose X-FLoRA, a cross-modal FL framework that uses modality-expert low-rank adaptation (LoRA) for medical VQA. Specifically, X-FLoRA enables the synthesis of images from one modality to another without requiring data sharing between clients. This is achieved by training a backward translation model within a federated asymmetric translation scheme that integrates clinical semantics from textual data. Additionally, X-FLoRA introduces modality-expert LoRA, which fine-tunes separate LoRA modules to strengthen modality-specific representations in the VQA task. The server aggregates the trained backward translation models and fine-tuned LoRA modules using discriminator quality scores and expert-aware weighting, which regulate the relative contributions from different clients. Experiments were conducted on VQA datasets encompassing different medical modalities, and the results demonstrate that X-FLoRA outperforms existing FL methods in terms of VQA performance.