Jaehyeok Lee
Author directory2026
Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages
Tarek Naous | Anagha Savit | Carlos Rafael Catalan | Geyang Guo | Jaehyeok Lee | Kyungdon Lee | Lheane Marie Dizon | Mengyu Ye | Neel Kothari | Sahajpreet Singh | Sarah Masud | Tanish Patwa | Trung Thanh Tran | Zohaib Khan | Alan Ritter | Tanmoy Chakraborty | Yuki Arase | Keisuke Sakaguchi | JinYeong Bak | Wei Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Tarek Naous | Anagha Savit | Carlos Rafael Catalan | Geyang Guo | Jaehyeok Lee | Kyungdon Lee | Lheane Marie Dizon | Mengyu Ye | Neel Kothari | Sahajpreet Singh | Sarah Masud | Tanish Patwa | Trung Thanh Tran | Zohaib Khan | Alan Ritter | Tanmoy Chakraborty | Yuki Arase | Keisuke Sakaguchi | JinYeong Bak | Wei Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
As Large Language Models (LLMs) develop stronger multilingual capabilities, their sensitivity to culturally diverse entities becomes increasingly important. Prior work by Naous et al. (2024) has shown that LLMs often favor Western-associated entities in Arabic. Due to the lack of entity-centric multilingual benchmarks, it remains unclear if such biases also manifest in various non-Western languages. In this paper, we introduce Camellia, a benchmark for evaluating entity-centric cultural biases in nine Asian languages, spanning six Asian cultures. Camellia includes 19,530 manually annotated entities associated with the covered Asian or Western cultures, as well as 2,173 masked contexts for these entities derived from social media posts. Using Camellia, we evaluate cultural biases in four recent multilingual LLMs across three tasks: cultural context adaptation, sentiment association, and entity extractive QA. Our analyses show that LLMs struggle with cultural adaptation across these languages, with performance differing across models developed in different regions. We further observe that different LLM families can hold distinct biases, reflected in the ways they link cultures to particular sentiments. Lastly, we find that LLMs can struggle with context understanding in some Asian languages, creating performance gaps between cultures in entity extraction.
2025
Self-Training Meets Consistency: Improving LLMs’ Reasoning with Consistency-Driven Rationale Evaluation
Jaehyeok Lee | Keisuke Sakaguchi | JinYeong Bak
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Jaehyeok Lee | Keisuke Sakaguchi | JinYeong Bak
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Self-training approach for large language models (LLMs) improves reasoning abilities by training the models on their self-generated rationales. Previous approaches have labeled rationales that produce correct answers for a given question as appropriate for training. However, a single measure risks misjudging rationale quality, leading the models to learn flawed reasoning patterns. To address this issue, we propose CREST (Consistency-driven Rationale Evaluation for Self-Training), a self-training framework that further evaluates each rationale through follow-up questions and leverages this evaluation to guide its training. Specifically, we introduce two methods: (1) filtering out rationales that frequently result in incorrect answers on follow-up questions and (2) preference learning based on mixed preferences from rationale evaluation results of both original and follow-up questions. Experiments on three question-answering datasets using open LLMs show that CREST not only improves the logical robustness and correctness of rationales but also improves reasoning abilities compared to previous self-training approaches.
Unintended Harms of Value-Aligned LLMs: Psychological and Empirical Insights
Sooyung Choi | Jaehyeok Lee | Xiaoyuan Yi | Jing Yao | Xing Xie | JinYeong Bak
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Sooyung Choi | Jaehyeok Lee | Xiaoyuan Yi | Jing Yao | Xing Xie | JinYeong Bak
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
The application scope of Large Language Models (LLMs) continues to expand, leading to increasing interest in personalized LLMs that align with human values. However, aligning these models with individual values raises significant safety concerns, as certain values may correlate with harmful information. In this paper, we identify specific safety risks associated with value-aligned LLMs and investigate the psychological principles behind these challenges. Our findings reveal two key insights. (1) Value-aligned LLMs are more prone to harmful behavior compared to non-fine-tuned models and exhibit slightly higher risks in traditional safety evaluations than other fine-tuned models. (2) These safety issues arise because value-aligned LLMs genuinely generate text according to the aligned values, which can amplify harmful outcomes. Using a dataset with detailed safety categories, we find significant correlations between value alignment and safety risks, supported by psychological hypotheses. This study offers insights into the “black box” of value alignment and proposes in-context alignment methods to enhance the safety of value-aligned LLMs.
Search