Jaeyoung Lee
Author directoryOther people with similar names: Jaeyoung Lee
Unverified author pages with similar names: Jaeyoung Lee
2026
Investigating Counterfactual Unfairness in LLMs towards Identities through Humor
Shubin Kim | Yejin Son | Junyeong Park | Keummin Ka | Seungbeen Lee | Jaeyoung Lee | Hyeju Jang | Alice Oh | Youngjae Yu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Shubin Kim | Yejin Son | Junyeong Park | Keummin Ka | Seungbeen Lee | Jaeyoung Lee | Hyeju Jang | Alice Oh | Youngjae Yu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Humor holds up a mirror to social perception: what we find funny often reflects who we are and how we judge others. When language models engage with humor, their reactions expose the social assumptions they have internalized from training data. In this paper, we investigate counterfactual unfairness through humor by observing how the model’s responses change when we swap who speaks and who is addressed while holding other factors constant. Our framework spans three tasks: humor generation refusal, speaker intention inference, and relational/societal impact prediction, covering both identity-agnostic humor and identity-specific disparagement humor. We introduce interpretable bias metrics that capture asymmetric patterns under identity swaps. Experiments across state-of-the-art models reveal consistent relational disparities: jokes told by privileged speakers are refused up to 67.5% more often, judged as malicious 64.7% more frequently, and rated up to 1.5 points higher in social harm on a 5-point scale. These patterns highlight how sensitivity and stereotyping coexist in generative models, complicating efforts toward fairness and cultural alignment.
2024
How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models
Jaeyoung Lee | Ximing Lu | Jack Hessel | Faeze Brahman | Youngjae Yu | Yonatan Bisk | Yejin Choi | Saadia Gabriel
Findings of the Association for Computational Linguistics: EMNLP 2024
Jaeyoung Lee | Ximing Lu | Jack Hessel | Faeze Brahman | Youngjae Yu | Yonatan Bisk | Yejin Choi | Saadia Gabriel
Findings of the Association for Computational Linguistics: EMNLP 2024
Given the growing influx of misinformation across news and social media, there is a critical need for systems that can provide effective real-time verification of news claims. Large language or multimodal model based verification has been proposed to scale up online policing mechanisms for mitigating spread of false and harmful content. While these can potentially reduce burden on human fact-checkers, such efforts may be hampered by foundation model training data becoming outdated. In this work, we test the limits of improving foundation model performance without continual updating through an initial study of knowledge transfer using either existing intra- and inter-domain benchmarks or explanations generated from large language models (LLMs).We evaluate on 12 public benchmarks for fact-checking and misinformation detection as well as two other tasks relevant to content moderation - toxicity and stance detection. Our results on two recent multi-modal fact-checking benchmarks, Mocheg and Fakeddit, indicate that knowledge transfer strategies can improve Fakeddit performance over the state-of-the-art by up to 1.7% and Mocheg performance by up to 2.9%. The code, model checkpoints, and dataset are available: https://github.com/given131/ fact-verifier-knowledge-transfer.