Taegyoon Kim
Author directory2026
Who Counts as "the People"? Measuring Contested Political Concepts from Political Speech using Masked Language Models
Hyeongjae Lee | Seonwoo Ahn | Sunwoo Kwak | June Jeon | Jisung Yoon | Taegyoon Kim
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Hyeongjae Lee | Seonwoo Ahn | Sunwoo Kwak | June Jeon | Jisung Yoon | Taegyoon Kim
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
What do political elites mean when they invoke "the people", and how has this meaning shifted across centuries of democratic history? Political theorists have long held that “the people” is a discursively constructed and contested category. Yet empirical work remains largely qualitative or small in scale, and computational approaches have yet to address how contested concepts themselves are framed over time. We address this gap with a generalizable framework that models contested concepts as configurations of theoretically grounded frames, and apply it to American presidential rhetoric based on three theoretically grounded frames: Nation, Civic, and Global. Leveraging a fine-tuned political masked language model, we mask each occurrence of "people" and score each frame by the pseudo-log-likelihood of its probe words. We apply the framework to 3,402 U.S. presidential speeches from 1789 to 2025. The resulting measurements agree with expert human coding at a level comparable to inter-annotator agreement, while tracking major junctures such as the Civil War, the World Wars, the Civil Rights era, and a recent nationalist turn. They also covary with external indicators as expected: stronger democracy is associated with less national and more civic framing. Beyond "the people", the framework can generalize to other contested political concepts.
Can LLMs Reliably Self-Report Adversarial Prefills, and How?
Quang Minh Nguyen | Uzair Ahmed | Taegyoon Kim
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Quang Minh Nguyen | Uzair Ahmed | Taegyoon Kim
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Prior work shows that large language models (LLMs) exhibit varying degrees of introspective capability on benign tasks. We extend the question to safety contexts and examine how reliably a model can recognize that its own prior response was elicited by an adversarial prefill attack. Across ten open-weight instruction-tuned LLMs from 3B to 70B parameters and four safety benchmarks, no model reliably recognizes its own compromised outputs, with models claiming intent on prefilled responses at an average rate of 25.3%. Introspective signal stems primarily from reasoning about safety and refusal. Orthogonalizing models’ weights against the refusal direction collapses the gap between claim rates on prefilled and natural outputs to near zero, though the direction is not its unique mediator. Framing the question as internal intention versus external tampering elicits qualitatively different responses on the same models. Training models to mimic correct introspective answers or optimize an introspective objective can improve the accuracy of introspection, but such training does not transfer to the tampering probe and counterintuitively raises attack success rate under adversarial prefill on most models, amounting to a partial mitigation. These findings outline mechanisms underpinning the observed introspective signals in safety contexts and highlight risks in the reliability of LLM self-reports
2025
Whose Palestine Is It? A Topic Modelling Approach to National Framing in Academic Research
Maida Aizaz | Taegyoon Kim | Lanu Kim
Proceedings of the 9th Widening NLP Workshop
Maida Aizaz | Taegyoon Kim | Lanu Kim
Proceedings of the 9th Widening NLP Workshop
In this study, we investigate how author affiliation shapes academic discourse, proposing it as an effective proxy for author perspective in understanding what topics are studied, how nations are framed, and whose realities are prioritised. Using Palestine as a case study, we apply BERTopic and Structural Topic Modelling (STM) to 29,536 English-language academic articles collected from the OpenAlex database. We find that domestic authors focus on practical, local issues like healthcare, education, and the environment, while foreign authors emphasise legal, historical, and geopolitical discussions. These differences, in our interpretation, reflect lived proximity to war and crisis. We also note that while BERTopic captures greater lexical nuance, STM enables covariate-aware comparisons, offering deeper insight into how affiliation correlates with thematic emphasis. We propose extending this framework to other underrepresented countries, including a future study focused on Gaza post-October 7.
Is External Information Useful for Stance Detection with LLMs?
Quang Minh Nguyen | Taegyoon Kim
Findings of the Association for Computational Linguistics: ACL 2025
Quang Minh Nguyen | Taegyoon Kim
Findings of the Association for Computational Linguistics: ACL 2025
In the stance detection task, a text is classified as either favorable, opposing, or neutral towards a target. Prior work suggests that the use of external information, e.g., excerpts from Wikipedia, improves stance detection performance. However, whether or not such information can benefit large language models (LLMs) remains an unanswered question, despite their wide adoption in many reasoning tasks. In this study, we conduct a systematic evaluation on how Wikipedia and web search external information can affect stance detection across eight LLMs and in three datasets with 12 targets. Surprisingly, we find that such information degrades performance in most cases, with macro F1 scores dropping by up to 27.9%. We explain this through experiments showing LLMs’ tendency to align their predictions with the stance and sentiment of the provided information rather than the ground truth stance of the given text. We also find that performance degradation persists with chain-of-thought prompting, while fine-tuning mitigates but does not fully eliminate it. Our findings, in contrast to previous literature on BERT-based systems which suggests that external information enhances performance, highlight the risks of information biases in LLM-based stance classifiers.