Il-Youp Kwak
Author directory2026
KMMMU: Evaluation of Massive Multi-discipline Multimodal Understanding in Korean Language and Context
Nahyun Lee | Guijin Son | Hyunwoo Ko | Chanyoung Kim | JunYoung An | Kyubeen Han | Youngjae Yu | Il-Youp Kwak
Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026)
Nahyun Lee | Guijin Son | Hyunwoo Ko | Chanyoung Kim | JunYoung An | Kyubeen Han | Youngjae Yu | Il-Youp Kwak
Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026)
Multimodal evaluation is strengthened when benchmarks reflect the languages, institutions, and disciplinary conventions that shape expert visual reasoning. We introduce KMMMU, a native Korean benchmark built from Korean-native examination materials. KMMMU contains 3,466 questions spanning nine disciplines and nine visual modality categories. For targeted evaluation on challenging questions and Korean-specific knowledge, we additionally provide KMMMU-Adv and KMMMU-Local, respectively. Experiments show that the strongest evaluated open-weight model reaches 42.05% on the KMMMU, while the best evaluated proprietary model achieves 52.42% on the KMMMU-Adv. Performance varies substantially across disciplines and answer formats, with Korean-specific questions showing gaps of up to 13.43%. Error analysis suggests that inspected failures often involve convention-to-label mapping, few-shot symbolic induction, localized knowledge recall, and domain-specific standards, not reasoning depth alone. KMMMU provides a testbed for native Korean expert-level multimodal understanding beyond English-centric or translation-based evaluation.