San Kim
Author directoryOther people with similar names: San Kim
Unverified author pages with similar names: San Kim
2026
KoEVD: A Korean Benchmark for Metadata-Conditioned Multi-Stage LLM Safety Evaluation
San Kim | Wonjung Kim | Heesoo Kwon | Jae Hee Yang
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
San Kim | Wonjung Kim | Heesoo Kwon | Jae Hee Yang
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Large language model (LLM) safety benchmarks often evaluate input risk, response selection, generation, and tool use separately, making comparisons across tasks difficult. We introduce KoEVD (Korean Ethical Violation Dataset), which links five evaluation and analysis tasks through annotated Korean source utterances: input-risk judgment, candidate-response selection, generation safety, response-strategy analysis, and non-executable mock-tool selection. The tasks share source IDs, with task-specific subsets and candidate sampling where required. Built through an approximately 3.5-year human-governed, AI-assisted process, KoEVD contains 13,552 curated user utterances and 71,395 linked response candidates. Across 12 open-weight and commercial models, observed task profiles differ, and aggregate scores obscure variation across severity, explicitness, utterance type, and topic. In matched generation and mock-tool-selection cases, responses classified as safe coexist with unsafe mock-tool choices; pooled unsafe-call rates are 9.73% and 9.31% for the classifier-safe and classifier-unsafe groups, respectively, with framing-specific differences. These diagnostic measurements do not establish real-world safety. KoEVD supports detailed Korean safety analysis through contextual annotations and source-linked comparisons.
KoMMERCE: A Korean Multimodal and Multitask Evaluation Benchmark for Reliable Conversational E-commerce Agents
Byunggill Joe | San Kim | Minyoung Jung | Dong Hyok Shin | Yu-Jeong Sung | Saim Shin
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Byunggill Joe | San Kim | Minyoung Jung | Dong Hyok Shin | Yu-Jeong Sung | Saim Shin
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
E-commerce conversations are increasingly multimodal, yet end-to-end response scores cannot localize which dialogue competence failed, and no public benchmark evaluates finegrained multimodal dialogue understanding for Korean commercial agents. We present KoMMERCE, a Korean multimodal fashion-domain benchmark that coordinates nine fine-grained dialogue understanding tasks on the same conversations: sentiment, intent, and strategy classification;image and text knowledge retrieval;dialogue state tracking; span detection; coreference resolution; and the first formulation of cross-modal dialogue relation extraction. The corpus contains 15,293 dyadic role-play dialogues under two-stage quality control. Baselines on six frontier multimodal LLMs reveal task-conditional patterns that should inform deployment choices: no model wins all nine tasks, and context length affects accuracy differently from one task to another.