Sangho Kim
Author directory2026
Thunder-NUBench: A Benchmark for LLMs’ Sentence-Level Negation Understanding
Yeonkyoung So | Gyuseong Lee | Sungmok Jung | Joonhak Lee | JiA Kang | Sangho Kim | Jaejin Lee
Findings of the Association for Computational Linguistics: EACL 2026
Yeonkyoung So | Gyuseong Lee | Sungmok Jung | Joonhak Lee | JiA Kang | Sangho Kim | Jaejin Lee
Findings of the Association for Computational Linguistics: EACL 2026
Negation is a fundamental linguistic phenomenon that poses ongoing challenges for Large Language Models (LLMs), particularly in tasks requiring deep semantic understanding. Current benchmarks often treat negation as a minor detail within broader tasks, such as natural language inference. Consequently, there is a lack of benchmarks specifically designed to evaluate comprehension of negation. In this work, we introduce Thunder-NUBench — a novel benchmark explicitly created to assess sentence-level understanding of negation in LLMs. Thunder-NUBench goes beyond identifying surface-level cues by contrasting standard negation with structurally diverse alternatives, such as local negation, contradiction, and paraphrase. This benchmark includes manually created sentence-negation pairs and a multiple-choice dataset, allowing for a comprehensive evaluation of models’ understanding of negation.
Thunder-KoNUBench: A Corpus-Aligned Benchmark for Korean Negation Understanding
Sungmok Jung | Yeonkyoung So | Joonhak Lee | Sangho Kim | Yelim Ahn | Jaejin Lee
Findings of the Association for Computational Linguistics: ACL 2026
Sungmok Jung | Yeonkyoung So | Joonhak Lee | Sangho Kim | Yelim Ahn | Jaejin Lee
Findings of the Association for Computational Linguistics: ACL 2026
Although negation is known to challenge large language models (LLMs), benchmarks for evaluating negation understanding—especially in Korean—are scarce. We conduct a corpus-based analysis of Korean negation and show that LLM performance degrades under negation. We then introduce Thunder-KoNUBench, a sentence-level negation understanding benchmark that reflects the empirical distribution of Korean negation phenomena. Evaluating 47 LLMs on Thunder-KoNUBench, we analyze the effects of model size and instruction tuning, and perform error analysis to better understand model behavior. We further show that fine-tuning on Thunder-KoNUBench improves negation understanding and broader contextual comprehension in Korean.
Polar: A Benchmark for Evaluating Political Bias in LLMs
Sangho Kim | Heejin Kim | Hyunggeun Jeon | Yoonhee Park | Jaejin Lee
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Sangho Kim | Heejin Kim | Hyunggeun Jeon | Yoonhee Park | Jaejin Lee
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Political bias in large language models (LLMs) is increasingly significant, but difficult to measure reproducibly across political and linguistic contexts. We introduce Polar, a 4,026-instance multiple-choice benchmark that measures political bias through option-level likelihoods rather than prompt-based generation. Polar covers two ideological axes and eight issue categories derived from the Manifesto Project, and evaluates models in parallel across U.S. and South Korean political contexts. Across 38 LLMs, measured bias varies systematically with political context, issue category, model group, and presentation language. All models lean left-progressive on U.S. political content, but show more centered and mixed patterns on South Korean content. Translation experiments further show that presentation language alone can shift measured bias. These findings highlight the need for multilingual and cross-contextual evaluation of political bias in LLMs.
Thunder-DeID+: Boundary-Aware Korean De-identification from Court Judgments to Broader Sensitive Documents
Gyuseong Lee | Gyeongje Cho | Jongmin Kim | Heejin Kim | Yeonkyoung So | Hyunggeun Jeon | Hyunji M. Park | Sangho Kim | Yoonhee Park | Youngbin Pyo | Jaejin Lee
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Gyuseong Lee | Gyeongje Cho | Jongmin Kim | Heejin Kim | Yeonkyoung So | Hyunggeun Jeon | Hyunji M. Park | Sangho Kim | Yoonhee Park | Youngbin Pyo | Jaejin Lee
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
This study examines the transferability of a de-identification model developed for Korean court judgments to a broader range of sensitive documents, focusing on boundary precision, label granularity, and recall. The agglutinative structure of Korean causes Byte Pair Encoding (BPE) tokenizers to frequently merge Personally Identifiable Information (PII) with adjacent particles, which complicates accurate span recovery. Furthermore, current PII taxonomies lack sufficient granularity for legal texts. To address these issues, this study introduces Thunder-DeID+, which integrates a refined three-tier PII taxonomy with a boundary-aware tokenizer. The tokenizer enhances a pretrained BPE vocabulary by incorporating N-gram and boundary-scoring components to better align token and PII span boundaries. The proposed approach is evaluated using a tokenizer-independent word-segment-level protocol. On full-length judgments, the tokenizer consistently improves W. Strict Typed F1-score from 0.76–0.77 to 0.87–0.88 across four model sizes (150M, 395M, 750M, and 1B), mainly by increasing precision rather than model scale. With fine-tuning on 1,000 broader sensitive documents, the judgment-trained model further achieves 0.93 W. Strict Typed F1-score, indicating effective transfer beyond court judgments.
KBS: A Korean Benchmark Suite for Foundation Model Evaluation
Yeonkyoung So | Jongmin Kim | Sungmok Jung | Gyuseong Lee | Sangho Kim | Jongyeon Park | Joonhak Lee | Seho Pyo | Gyeongje Cho | Seorin Kim | Jisoo Kim | Suyoung Park | Hyunji M. Park | Yelim Ahn | Yeongho Seo | Jaejin Lee
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Yeonkyoung So | Jongmin Kim | Sungmok Jung | Gyuseong Lee | Sangho Kim | Jongyeon Park | Joonhak Lee | Seho Pyo | Gyeongje Cho | Seorin Kim | Jisoo Kim | Suyoung Park | Hyunji M. Park | Yelim Ahn | Yeongho Seo | Jaejin Lee
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
We present a Korean Benchmark Suite (KBS) comprising Ko-ARC, Ko-GSM8K, Ko-EQ-Bench, Ko-WinoGrande, Ko-LAMBADA, and Ko-IFEval, covering six capabilities across 9,396 items. The suite combines review, localization, direct Korean construction, and redesign for Korean, avoiding a translation-only approach to benchmark adaptation. We evaluate Korean-developed models, multilingual open models, and closed API models under protocols that account for model access, prompt format, and scoring differences. Results show that Korean benchmark scores depend on task construction, scoring rules, prompting, and model post-training. These findings suggest that Korean evaluation should report both final scores and the settings used to obtain them.