JinYeong Bak
Author directoryAlso published as: Jinyeong Bak
2026
Proceedings of the 11th Workshop on Natural User-generated Text
JinYeong Bak | Rob van der Goot | Hyeju Jang | Weerayut Buaphet | Alan Ramponi | Wei Xu | Alan Ritter
Proceedings of the 11th Workshop on Natural User-generated Text
JinYeong Bak | Rob van der Goot | Hyeju Jang | Weerayut Buaphet | Alan Ramponi | Wei Xu | Alan Ritter
Proceedings of the 11th Workshop on Natural User-generated Text
Lost in Mimicry: Verified Training-Free Post-Editing for LLM Translationese
Dongwon Noh | Donghyeok Koh | Yeon-Soo Lim | Seohyeong Jeong | Yunsu Kim | Gyuwan Kim | Soojong Do | Jinyeong Bak | Sugyeong Eo | Cheoneum Park
Proceedings of the Eleventh Conference on Machine Translation
Dongwon Noh | Donghyeok Koh | Yeon-Soo Lim | Seohyeong Jeong | Yunsu Kim | Gyuwan Kim | Soojong Do | Jinyeong Bak | Sugyeong Eo | Cheoneum Park
Proceedings of the Eleventh Conference on Machine Translation
Translations generated by Large Language Models (LLMs) often exhibit unnatural stylistic traces distinct from human-written translations, which we refer to as LLM translationese. We propose \textit{MELT}, a training-free post-editing framework for mitigating LLM translationese across five target languages (Korean, Chinese, Japanese, German, and French) without weight updates. MELT defines translationese patterns based on faithfulness, voice, register, naturalness, and surface form, and combines a three-stage verification gate with selective multi-agent debate to preserve meaning and mitigate self-bias. Experiments on FLORES-200 show that verification contributes most to quality improvement and that MELT achieves the lowest calibration error with respect to Oracle preference across the five languages. Our analysis further shows that LLM translationese exhibits language-specific realizations while sharing a common mechanism of English-structure mimicry.
CoCoA: Context-Conditional Cultural Alignment for Large Language Models
Kyungdon Lee | Wei Xu | Alan Ritter | Dong-Ho Lee | JinYeong Bak
Findings of the Association for Computational Linguistics: EMNLP 2026
Kyungdon Lee | Wei Xu | Alan Ritter | Dong-Ho Lee | JinYeong Bak
Findings of the Association for Computational Linguistics: EMNLP 2026
Large Language Models (LLMs) often favor Western-associated entities across cultural contexts. Conventional debiasing methods aim for uniform neutrality, but cultural bias mitigation demands context-conditional behavior, preferring culturally appropriate entities when cultural cues are present and remaining neutral when they are absent. We propose CoCoA (Context-Conditional Cultural Alignment), a framework that learns this behavior through dual-context training on the same entity pairs under contexts with and without cultural cues. CoCoA combines a contrastive alignment objective with calibration and drift regularization, optimized through goal-aware gradient reconciliation. We evaluate CoCoA on CAMeL and Camellia, two entity-centric cultural bias benchmarks, across ten language settings and four LLMs. CoCoA reduces the Cultural Bias Score from 43 to 24 on average while maintaining near-neutral preferences at 50.2, with minimal impact on general performance across five standard benchmarks. These findings highlight that effective cultural alignment requires context-conditional modeling rather than uniform debiasing, and establish a new direction for mitigating entity-centric cultural bias in LLMs.
CHARM: Character Hallucination for Multicultural Role Play Benchmark
Sunkyung Han | Nahyeon Park | Gaeun Seo | Seunghyun Yoon | JinYeong Bak
Findings of the Association for Computational Linguistics: EMNLP 2026
Sunkyung Han | Nahyeon Park | Gaeun Seo | Seunghyun Yoon | JinYeong Bak
Findings of the Association for Computational Linguistics: EMNLP 2026
Role-playing large language models (LLMs) are expected to adopt a character’s style while also respecting that character’s knowledge boundaries. Prior evaluations detect character hallucination but rarely distinguish whether errors arise from failure to recognize a boundary or from failure to comply despite recognition. We introduce CHARM, a multicultural benchmark of 40 real and fictional characters drawn from five cultural-linguistic regions, and validated by native reviewers. It probes two boundary types, Temporal (historical vs. modern) and Cross-Universe (entities outside a character’s narrative or historical universe), using abstention-enabled multiple-choice questions. We propose a two-stage evaluation that separates Boundary-Awareness (explicit recognition that a query is out of scope) from Boundary-Compliance (abstention when answering concrete questions). Evaluations across six LLMs show that hallucination is driven predominantly by compliance failures. Models frequently acknowledge that a query lies outside the character’s knowledge yet still provide factual, out-of-character answers. By re-posing the same questions to the target character, we confirm that a large fraction of these cases are verified parametric overrides; the model stores the relevant fact but fails to suppress it. We also observe systematic cultural variation in these failures, consistent with imbalances in how characters from different regions are represented in model knowledge.
PTCG: Persona-guided Tree-based Counterargument Generation
Eunbeen Son | Yohan Jo | Joonsuk Park | JinYeong Bak
Findings of the Association for Computational Linguistics: EMNLP 2026
Eunbeen Son | Yohan Jo | Joonsuk Park | JinYeong Bak
Findings of the Association for Computational Linguistics: EMNLP 2026
The ability to generate counterarguments is important for critical thinking and balanced discourse, yet existing approaches typically produce only a single counterargument, failing to capture the diversity and persuasiveness required in real-world debates. To address this limitation, we propose Persona-guided Tree-based Counterargument Generation (PTCG), a framework that combines Tree-of-Thoughts–inspired step-wise generation and pruning with speaker persona selection. By estimating the author’s persona from the original argument and incorporating speaker personas representing distinct perspectives, PTCG operationalizes perspective-taking and enables the generation of diverse counterarguments. Results from LLM-as-a-Judge, classifier-based assessment, and human evaluations indicate that PTCG shows consistent improvements in both the diversity and persuasiveness of counterarguments compared to baseline methods.
CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia
Bryan Chen Zhengyu Tan | Weihua Zheng | Thong T. Doan | Bich Ngoc Doan | Jia Wang Peh | Xiaoyuan Yi | Jing Yao | Xing Xie | Nancy F. Chen | Zhengyuan Liu | JinYeong Bak | Wafi Shamdi | Soo Kai Chie | Liew Yu Siong | Aina Azyyati Binti Mohamad Rezal | Lew Yan Yan Vanessa | Huadan Wu | Dylan Raharja | Nadya Yuki Wangsajaya | Akane Fukushige | Kazushi Kato | Koji Inoue | Tatsuya Kawahara | Jaehyung Seo | Dongjun Kim | Seungyoon Lee | Zi Haur Pang | Rui Yang Tan | Charibeth Ko Cheng | Maria Regina Justina Estuar | Jann Railey Montalan | Pham Minh Duc | Roy Ka-Wei Lee
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Bryan Chen Zhengyu Tan | Weihua Zheng | Thong T. Doan | Bich Ngoc Doan | Jia Wang Peh | Xiaoyuan Yi | Jing Yao | Xing Xie | Nancy F. Chen | Zhengyuan Liu | JinYeong Bak | Wafi Shamdi | Soo Kai Chie | Liew Yu Siong | Aina Azyyati Binti Mohamad Rezal | Lew Yan Yan Vanessa | Huadan Wu | Dylan Raharja | Nadya Yuki Wangsajaya | Akane Fukushige | Kazushi Kato | Koji Inoue | Tatsuya Kawahara | Jaehyung Seo | Dongjun Kim | Seungyoon Lee | Zi Haur Pang | Rui Yang Tan | Charibeth Ko Cheng | Maria Regina Justina Estuar | Jann Railey Montalan | Pham Minh Duc | Roy Ka-Wei Lee
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common use case: users seeking practical help over multiple turns in culturally grounded scenarios. We introduce CultureConverse, a scalable, multilingual simulation and evaluation harness for culturally grounded assistant dialogue that covers 10 East and Southeast Asian regions, 58 subgroup identities, and 7 domains. Each simulated and evaluated episode produces a scored interaction where the assistant assists the user and infers cultural constraints from partial information. The resulting CultureConverse-DS dataset contains 14,610 benchmark (evaluation) episodes and 274,295 oracle-guided (gold-mode) dialogues. In our benchmark evaluation of 18 models, GPT-5 mini achieves the highest assistance quality. Human annotation experiments suggest that our evaluation framework is a sufficient proxy for human judgment. Performance gains from fine-tuning on 27,860 high-quality CultureConverse-DS samples improve in-domain assistance and transfer out-of-domain to cultural MCQ and safety classification benchmarks. We release the harness, both splits, and judge prompts to support interactive evaluation of cultural competency.
Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages
Tarek Naous | Anagha Savit | Carlos Rafael Catalan | Geyang Guo | Jaehyeok Lee | Kyungdon Lee | Lheane Marie Dizon | Mengyu Ye | Neel Kothari | Sahajpreet Singh | Sarah Masud | Tanish Patwa | Trung Thanh Tran | Zohaib Khan | Alan Ritter | Tanmoy Chakraborty | Yuki Arase | Keisuke Sakaguchi | JinYeong Bak | Wei Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Tarek Naous | Anagha Savit | Carlos Rafael Catalan | Geyang Guo | Jaehyeok Lee | Kyungdon Lee | Lheane Marie Dizon | Mengyu Ye | Neel Kothari | Sahajpreet Singh | Sarah Masud | Tanish Patwa | Trung Thanh Tran | Zohaib Khan | Alan Ritter | Tanmoy Chakraborty | Yuki Arase | Keisuke Sakaguchi | JinYeong Bak | Wei Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
As Large Language Models (LLMs) develop stronger multilingual capabilities, their sensitivity to culturally diverse entities becomes increasingly important. Prior work by Naous et al. (2024) has shown that LLMs often favor Western-associated entities in Arabic. Due to the lack of entity-centric multilingual benchmarks, it remains unclear if such biases also manifest in various non-Western languages. In this paper, we introduce Camellia, a benchmark for evaluating entity-centric cultural biases in nine Asian languages, spanning six Asian cultures. Camellia includes 19,530 manually annotated entities associated with the covered Asian or Western cultures, as well as 2,173 masked contexts for these entities derived from social media posts. Using Camellia, we evaluate cultural biases in four recent multilingual LLMs across three tasks: cultural context adaptation, sentiment association, and entity extractive QA. Our analyses show that LLMs struggle with cultural adaptation across these languages, with performance differing across models developed in different regions. We further observe that different LLM families can hold distinct biases, reflected in the ways they link cultures to particular sentiments. Lastly, we find that LLMs can struggle with context understanding in some Asian languages, creating performance gaps between cultures in entity extraction.
Q-TIE: A Lightweight and Generalizable Re-ranking Framework for Temporal Information Retrieval
Soyeon Kim | HyunJin Kim | JinYeong Bak | Steven Euijong Whang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Soyeon Kim | HyunJin Kim | JinYeong Bak | Steven Euijong Whang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Temporal Information Retrieval (TIR) has been increasingly critical given the rise of Retrieval-Augmented Generation (RAG). Since temporally mismatched evidence can be highly misleading, TIR aims to retrieve documents that are both semantically and temporally relevant to a query. Two TIR paradigms have emerged – temporal retrievers and temporal re-rankers – differing in how temporal relevance is modeled. While these paradigms provide complementary strengths, our analysis reveals that each alone falls short of robust TIR: temporal retrievers provide flexible query understanding via learned representations, but often fail to explicitly account for temporal constraints; temporal re-rankers can enforce such constraints more explicitly, but often rely on predefined re-ranking rules. To address this, we propose Q-TIE, a re-ranking framework based on learned Temporal Intent Extraction (TIE). By introducing a TIE model that maps each query’s temporal constraint into a unified interval representation (i.e., ⟨ tstart, tend ⟩), Q-TIE generalizes beyond predefined rules via model-based learning while explicitly modeling temporal constraints as a separate signal – jointly achieving what each paradigm typically trades off. Experiments demonstrate that Q-TIE consistently outperforms existing TIR methods with stronger generalizability across temporal query types, and provides a lightweight yet effective add-on for temporally-aware RAG pipelines.
2025
Proceedings of the Tenth Workshop on Noisy and User-generated Text
JinYeong Bak | Rob van der Goot | Hyeju Jang | Weerayut Buaphet | Alan Ramponi | Wei Xu | Alan Ritter
Proceedings of the Tenth Workshop on Noisy and User-generated Text
JinYeong Bak | Rob van der Goot | Hyeju Jang | Weerayut Buaphet | Alan Ramponi | Wei Xu | Alan Ritter
Proceedings of the Tenth Workshop on Noisy and User-generated Text
Tagged Span Annotation for Detecting Translation Errors in Reasoning LLMs
Taemin Yeom | Yonghyun Ryu | Yoonjung Choi | Jinyeong Bak
Proceedings of the Tenth Conference on Machine Translation
Taemin Yeom | Yonghyun Ryu | Yoonjung Choi | Jinyeong Bak
Proceedings of the Tenth Conference on Machine Translation
We present the AIP team’s submission to the WMT 2025 Unified MT Evaluation SharedTask, focusing on the span-level error detection subtask. Our system emphasizes response format design to better harness the capabilities of OpenAI’s o3, the state-of-the-art reasoning LLM. To this end, we introduce Tagged SpanAnnotation (TSA), an annotation scheme designed to more accurately extract span-level information from the LLM. On our refined version of WMT24 ESA dataset, our reference-free method achieves an F1 score of approximately 27 for character-level label prediction, outperforming the reference-based XCOMET-XXL at approximately 17.
Text Overlap: An LLM with Human-like Conversational Behaviors
JiWoo Kim | Minsuk Chang | JinYeong Bak
Proceedings of the Third Workshop on Social Influence in Conversations (SICon 2025)
JiWoo Kim | Minsuk Chang | JinYeong Bak
Proceedings of the Third Workshop on Social Influence in Conversations (SICon 2025)
Traditional text-based human-AI interactions typically follow a strict turn-taking approach. This rigid structure limits conversational flow, unlike natural human conversations, which can freely incorporate overlapping speech. However, our pilot study suggests that even in text-based interfaces, overlapping behaviors such as backchanneling and proactive responses lead to more natural and functional exchanges. Motivated by these findings, we introduce text-based overlapping interactions as a new challenge in human-AI communication, characterized by real-time typing, diverse response types, and interruptions. To enable AI systems to handle such interactions, we define three core tasks: deciding when to overlap, selecting the response type, and generating utterances. We construct a synthetic dataset for these tasks and train OverlapBot, an LLM-driven chatbot designed to engage in text-based overlapping interactions. Quantitative and qualitative evaluations show that OverlapBot increases turn exchanges compared to traditional turn-taking systems, with users making 72% more turns and the chatbot 130% more turns, which is perceived as efficient by end-users. This finding supports overlapping interactions and enhances communicative efficiency and engagement.
Self-Training Meets Consistency: Improving LLMs’ Reasoning with Consistency-Driven Rationale Evaluation
Jaehyeok Lee | Keisuke Sakaguchi | JinYeong Bak
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Jaehyeok Lee | Keisuke Sakaguchi | JinYeong Bak
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Self-training approach for large language models (LLMs) improves reasoning abilities by training the models on their self-generated rationales. Previous approaches have labeled rationales that produce correct answers for a given question as appropriate for training. However, a single measure risks misjudging rationale quality, leading the models to learn flawed reasoning patterns. To address this issue, we propose CREST (Consistency-driven Rationale Evaluation for Self-Training), a self-training framework that further evaluates each rationale through follow-up questions and leverages this evaluation to guide its training. Specifically, we introduce two methods: (1) filtering out rationales that frequently result in incorrect answers on follow-up questions and (2) preference learning based on mixed preferences from rationale evaluation results of both original and follow-up questions. Experiments on three question-answering datasets using open LLMs show that CREST not only improves the logical robustness and correctness of rationales but also improves reasoning abilities compared to previous self-training approaches.
Unintended Harms of Value-Aligned LLMs: Psychological and Empirical Insights
Sooyung Choi | Jaehyeok Lee | Xiaoyuan Yi | Jing Yao | Xing Xie | JinYeong Bak
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Sooyung Choi | Jaehyeok Lee | Xiaoyuan Yi | Jing Yao | Xing Xie | JinYeong Bak
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
The application scope of Large Language Models (LLMs) continues to expand, leading to increasing interest in personalized LLMs that align with human values. However, aligning these models with individual values raises significant safety concerns, as certain values may correlate with harmful information. In this paper, we identify specific safety risks associated with value-aligned LLMs and investigate the psychological principles behind these challenges. Our findings reveal two key insights. (1) Value-aligned LLMs are more prone to harmful behavior compared to non-fine-tuned models and exhibit slightly higher risks in traditional safety evaluations than other fine-tuned models. (2) These safety issues arise because value-aligned LLMs genuinely generate text according to the aligned values, which can amplify harmful outcomes. Using a dataset with detailed safety categories, we find significant correlations between value alignment and safety risks, supported by psychological hypotheses. This study offers insights into the “black box” of value alignment and proposes in-context alignment methods to enhance the safety of value-aligned LLMs.
2024
Proceedings of the Ninth Workshop on Noisy and User-generated Text (W-NUT 2024)
Rob van der Goot | JinYeong Bak | Max Müller-Eberstein | Wei Xu | Alan Ritter | Tim Baldwin
Proceedings of the Ninth Workshop on Noisy and User-generated Text (W-NUT 2024)
Rob van der Goot | JinYeong Bak | Max Müller-Eberstein | Wei Xu | Alan Ritter | Tim Baldwin
Proceedings of the Ninth Workshop on Noisy and User-generated Text (W-NUT 2024)
PEMA: An Offsite-Tunable Plug-in External Memory Adaptation for Language Models
HyunJin Kim | Young Jin Kim | JinYeong Bak
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
HyunJin Kim | Young Jin Kim | JinYeong Bak
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Pre-trained language models (PLMs) show impressive performance in various downstream NLP tasks. However, pre-training large language models demands substantial memory and training compute. Furthermore, due to the substantial resources required, many PLM weights are confidential. Consequently, users are compelled to share their data with model owners for fine-tuning specific tasks. To overcome the limitations, we introduce Plug-in External Memory Adaptation (PEMA), a Parameter-Efficient Fine-Tuning (PEFT) method, enabling PLM fine-tuning without requiring access to all the weights. PEMA integrates with context representations from test data during inference to perform downstream tasks. It uses external memory to store PLM-generated context representations mapped with target tokens. Our method utilizes weight matrices of LoRA-like bottlenecked adapter in the PLM’s final layer to enhance efficiency. Our approach also includes Gradual Unrolling, a novel interpolation strategy to improve generation quality. We validate PEMA’s effectiveness through experiments on syntactic and real datasets for machine translation and style transfer. Our findings show that PEMA outperforms other PEFT approaches in memory and latency efficiency for training, and also excels in maintaining sentence meaning and generating appropriate language and styles.
KpopMT: Translation Dataset with Terminology for Kpop Fandom
JiWoo Kim | Yunsu Kim | JinYeong Bak
Proceedings of the Seventh Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2024)
JiWoo Kim | Yunsu Kim | JinYeong Bak
Proceedings of the Seventh Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2024)
While machines learn from existing corpora, humans have the unique capability to establish and accept new language systems. This makes human form unique language systems within social groups. Aligning with this, we focus on a gap remaining in addressing translation challenges within social groups, where in-group members utilize unique terminologies. We propose KpopMT dataset, which aims to fill this gap by enabling precise terminology translation, choosing Kpop fandom as an initiative for social groups given its global popularity. Expert translators provide 1k English translations for Korean posts and comments, each annotated with specific terminology within social groups’ language systems. We evaluate existing translation systems including GPT models on KpopMT to identify their failure cases. Results show overall low scores, underscoring the challenges of reflecting group-specific terminologies and styles in translation. We make KpopMT publicly available.
2023
From Values to Opinions: Predicting Human Behaviors and Stances Using Value-Injected Large Language Models
Dongjun Kang | Joonsuk Park | Yohan Jo | JinYeong Bak
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Dongjun Kang | Joonsuk Park | Yohan Jo | JinYeong Bak
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Being able to predict people’s opinions on issues and behaviors in realistic scenarios can be helpful in various domains, such as politics and marketing. However, conducting large-scale surveys like the European Social Survey to solicit people’s opinions on individual issues can incur prohibitive costs. Leveraging prior research showing influence of core human values on individual decisions and actions, we propose to use value-injected large language models (LLM) to predict opinions and behaviors. To this end, we present Value Injection Method (VIM), a collection of two methods—argument generation and question answering—designed to inject targeted value distributions into LLMs via fine-tuning. We then conduct a series of experiments on four tasks to test the effectiveness of VIM and the possibility of using value-injected LLMs to predict opinions and behaviors of people. We find that LLMs value-injected with variations of VIM substantially outperform the baselines. Also, the results suggest that opinions and behaviors can be better predicted using value-injected LLMs than the baseline approaches.
It Ain’t Over: A Multi-aspect Diverse Math Word Problem Dataset
Jiwoo Kim | Youngbin Kim | Ilwoong Baek | JinYeong Bak | Jongwuk Lee
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Jiwoo Kim | Youngbin Kim | Ilwoong Baek | JinYeong Bak | Jongwuk Lee
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
The math word problem (MWP) is a complex task that requires natural language understanding and logical reasoning to extract key knowledge from natural language narratives. Previous studies have provided various MWP datasets but lack diversity in problem types, lexical usage patterns, languages, and annotations for intermediate solutions. To address these limitations, we introduce a new MWP dataset, named DMath (Diverse Math Word Problems), offering a wide range of diversity in problem types, lexical usage patterns, languages, and intermediate solutions. The problems are available in English and Korean and include an expression tree and Python code as intermediate solutions. Through extensive experiments, we demonstrate that the DMath dataset provides a new opportunity to evaluate the capability of large language models, i.e., GPT-4 only achieves about 75% accuracy on the DMath dataset.
Diversity Enhanced Narrative Question Generation for Storybooks
Hokeun Yoon | JinYeong Bak
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Hokeun Yoon | JinYeong Bak
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Question generation (QG) from a given context can enhance comprehension, engagement, assessment, and overall efficacy in learning or conversational environments. Despite recent advancements in QG, the challenge of enhancing or measuring the diversity of generated questions often remains unaddressed. In this paper, we introduce a multi-question generation model (mQG), which is capable of generating multiple, diverse, and answerable questions by focusing on context and questions. To validate the answerability of the generated questions, we employ a SQuAD 2.0 fine-tuned question answering model, classifying the questions as answerable or not. We train and evaluate mQG on the FairytaleQA dataset, a well-structured QA dataset based on storybooks, with narrative questions. We further apply a zero-shot adaptation on the TellMeWhy and SQuAD1.1 datasets. mQG shows promising results across various evaluation metrics, among strong baselines.
Conversational Emotion-Cause Pair Extraction with Guided Mixture of Experts
DongJin Jeong | JinYeong Bak
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
DongJin Jeong | JinYeong Bak
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Emotion-Cause Pair Extraction (ECPE) task aims to pair all emotions and corresponding causes in documents.ECPE is an important task for developing human-like responses. However, previous ECPE research is conducted based on news articles, which has different characteristics compared to dialogues. To address this issue, we propose a Pair-Relationship Guided Mixture-of-Experts (PRG-MoE) model, which considers dialogue features (e.g., speaker information).PRG-MoE automatically learns relationship between utterances and advises a gating network to incorporate dialogue features in the evaluation, yielding substantial performance improvement. We employ a new ECPE dataset, which is an English dialogue dataset, with more emotion-cause pairs in documents than news articles. We also propose Cause Type Classification that classifies emotion-cause pairs according to the types of the cause of a detected emotion. For reproducing the results, we make available all our code and data.
2022
HUE: Pretrained Model and Dataset for Understanding Hanja Documents of Ancient Korea
Haneul Yoo | Jiho Jin | Juhee Son | JinYeong Bak | Kyunghyun Cho | Alice Oh
Findings of the Association for Computational Linguistics: NAACL 2022
Haneul Yoo | Jiho Jin | Juhee Son | JinYeong Bak | Kyunghyun Cho | Alice Oh
Findings of the Association for Computational Linguistics: NAACL 2022
Historical records in Korea before the 20th century were primarily written in Hanja, an extinct language based on Chinese characters and not understood by modern Korean or Chinese speakers. Historians with expertise in this time period have been analyzing the documents, but that process is very difficult and time-consuming, and language models would significantly speed up the process. Toward building and evaluating language models for Hanja, we release the Hanja Understanding Evaluation dataset consisting of chronological attribution, topic classification, named entity recognition, and summary retrieval tasks. We also present BERT-based models continued training on the two major corpora from the 14th to the 19th centuries: the Annals of the Joseon Dynasty and Diaries of the Royal Secretariats. We compare the models with several baselines on all tasks and show there are significant improvements gained by training on the two corpora. Additionally, we run zero-shot experiments on the Daily Records of the Royal Court and Important Officials (DRRI). The DRRI dataset has not been studied much by the historians, and not at all by the NLP community.
Translating Hanja Historical Documents to Contemporary Korean and English
Juhee Son | Jiho Jin | Haneul Yoo | JinYeong Bak | Kyunghyun Cho | Alice Oh
Findings of the Association for Computational Linguistics: EMNLP 2022
Juhee Son | Jiho Jin | Haneul Yoo | JinYeong Bak | Kyunghyun Cho | Alice Oh
Findings of the Association for Computational Linguistics: EMNLP 2022
The Annals of Joseon Dynasty (AJD) contain the daily records of the Kings of Joseon, the 500-year kingdom preceding the modern nation of Korea.The Annals were originally written in an archaic Korean writing system, ‘Hanja’, and were translated into Korean from 1968 to 1993.The resulting translation was however too literal and contained many archaic Korean words; thus, a new expert translation effort began in 2012. Since then, the records of only one king have been completed in a decade.In parallel, expert translators are working on English translation, also at a slow pace and produced only one king’s records in English so far.Thus, we propose H2KE, a neural machine translation model, that translates historical documents in Hanja to more easily understandable Korean and to English.Built on top of multilingual neural machine translation, H2KE learns to translate a historical document written in Hanja, from both a full dataset of outdated Korean translation and a small dataset of more recently translated contemporary Korean and English.We compare our method against two baselines:a recent model that simultaneously learns to restore and translate Hanja historical documentand a Transformer based model trained only on newly translated corpora.The experiments reveal that our method significantly outperforms the baselines in terms of BLEU scores for both contemporary Korean and English translations.We further conduct extensive human evaluation which shows that our translation is preferred over the original expert translations by both experts and non-expert Korean speakers.
2021
Knowledge-Enhanced Evidence Retrieval for Counterargument Generation
Yohan Jo | Haneul Yoo | JinYeong Bak | Alice Oh | Chris Reed | Eduard Hovy
Findings of the Association for Computational Linguistics: EMNLP 2021
Yohan Jo | Haneul Yoo | JinYeong Bak | Alice Oh | Chris Reed | Eduard Hovy
Findings of the Association for Computational Linguistics: EMNLP 2021
Finding counterevidence to statements is key to many tasks, including counterargument generation. We build a system that, given a statement, retrieves counterevidence from diverse sources on the Web. At the core of this system is a natural language inference (NLI) model that determines whether a candidate sentence is valid counterevidence or not. Most NLI models to date, however, lack proper reasoning abilities necessary to find counterevidence that involves complex inference. Thus, we present a knowledge-enhanced NLI model that aims to handle causality- and example-based inference by incorporating knowledge graphs. Our NLI model outperforms baselines for NLI tasks, especially for instances that require the targeted inference. In addition, this NLI model further improves the counterevidence retrieval system, notably finding complex counterevidence better.
Learning Sequential and Structural Information for Source Code Summarization
YunSeok Choi | JinYeong Bak | CheolWon Na | Jee-Hyong Lee
Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021
YunSeok Choi | JinYeong Bak | CheolWon Na | Jee-Hyong Lee
Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021
2020
Speaker Sensitive Response Evaluation Model
JinYeong Bak | Alice Oh
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
JinYeong Bak | Alice Oh
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Automatic evaluation of open-domain dialogue response generation is very challenging because there are many appropriate responses for a given context. Existing evaluation models merely compare the generated response with the ground truth response and rate many of the appropriate responses as inappropriate if they deviate from the ground truth. One approach to resolve this problem is to consider the similarity of the generated response with the conversational context. In this paper, we propose an automatic evaluation model based on that idea and learn the model parameters from an unlabeled conversation corpus. Our approach considers the speakers in defining the different levels of similar context. We use a Twitter conversation corpus that contains many speakers and conversations to test our evaluation model. Experiments show that our model outperforms the other existing evaluation metrics in terms of high correlation with human annotation scores. We also show that our model trained on Twitter can be applied to movie dialogues without any additional training. We provide our code and the learned parameters so that they can be used for automatic evaluation of dialogue response generation models.
2019
Variational Hierarchical User-based Conversation Model
JinYeong Bak | Alice Oh
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
JinYeong Bak | Alice Oh
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Generating appropriate conversation responses requires careful modeling of the utterances and speakers together. Some recent approaches to response generation model both the utterances and the speakers, but these approaches tend to generate responses that are overly tailored to the speakers. To overcome this limitation, we propose a new model with a stochastic variable designed to capture the speaker information and deliver it to the conversational context. An important part of this model is the network of speakers in which each speaker is connected to one or more conversational partner, and this network is then used to model the speakers better. To test whether our model generates more appropriate conversation responses, we build a new conversation corpus containing approximately 27,000 speakers and 770,000 conversations. With this corpus, we run experiments of generating conversational responses and compare our model with other state-of-the-art models. By automatic evaluation metrics and human evaluation, we show that our model outperforms other models in generating appropriate responses. An additional advantage of our model is that it generates better responses for various new user scenarios, for example when one of the speakers is a known user in our corpus but the partner is a new user. For replicability, we make available all our code and data.
2018
Conversational Decision-Making Model for Predicting the King’s Decision in the Annals of the Joseon Dynasty
JinYeong Bak | Alice Oh
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
JinYeong Bak | Alice Oh
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Styles of leaders when they make decisions in groups vary, and the different styles affect the performance of the group. To understand the key words and speakers associated with decisions, we initially formalize the problem as one of predicting leaders’ decisions from discussion with group members. As a dataset, we introduce conversational meeting records from a historical corpus, and develop a hierarchical RNN structure with attention and pre-trained speaker embedding in the form of a, Conversational Decision Making Model (CDMM). The CDMM outperforms other baselines to predict leaders’ final decisions from the data. We explain why CDMM works better than other methods by showing the key words and speakers discovered from the attentions as evidence.
2017
Rotated Word Vector Representations and their Interpretability
Sungjoon Park | JinYeong Bak | Alice Oh
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Sungjoon Park | JinYeong Bak | Alice Oh
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Vector representation of words improves performance in various NLP tasks, but the high dimensional word vectors are very difficult to interpret. We apply several rotation algorithms to the vector representation of words to improve the interpretability. Unlike previous approaches that induce sparsity, the rotated vectors are interpretable while preserving the expressive performance of the original vectors. Furthermore, any prebuilt word vector representation can be rotated for improved interpretability. We apply rotation to skipgrams and glove and compare the expressive power and interpretability with the original vectors and the sparse overcomplete vectors. The results show that the rotated vectors outperform the original and the sparse overcomplete vectors for interpretability and expressiveness tasks.
2015
Five Centuries of Monarchy in Korea: Mining the Text of the Annals of the Joseon Dynasty
JinYeong Bak | Alice Oh
Proceedings of the 9th SIGHUM Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities (LaTeCH)
JinYeong Bak | Alice Oh
Proceedings of the 9th SIGHUM Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities (LaTeCH)
2014
Self-disclosure topic model for Twitter conversations
JinYeong Bak | Chin-Yew Lin | Alice Oh
Proceedings of the Joint Workshop on Social Dynamics and Personal Attributes in Social Media
JinYeong Bak | Chin-Yew Lin | Alice Oh
Proceedings of the Joint Workshop on Social Dynamics and Personal Attributes in Social Media
Self-disclosure topic model for classifying and analyzing Twitter conversations
JinYeong Bak | Chin-Yew Lin | Alice Oh
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
JinYeong Bak | Chin-Yew Lin | Alice Oh
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
2012
Search
Co-authors
- Alice Oh 11
- Alan Ritter 5
- Yohan Jo 3
- Jiwoo Kim 3
- Jaehyeok Lee 3
- Wei Xu 3
- Haneul Yoo 3
- Rob van der Goot 3
- Weerayut Buaphet 2
- Kyunghyun Cho 2
- Hyeju Jang 2
- Jiho Jin 2
- Hyunjin Kim 2
- Yunsu Kim 2
- Kyungdon Lee 2
- Chin-Yew Lin 2
- Joonsuk Park 2
- Alan Ramponi 2
- Keisuke Sakaguchi 2
- Juhee Son 2
- Wei Xu 2
- Jing Yao 2
- Xiaoyuan Yi 2
- Yuki Arase 1
- Ilwoong Baek 1
- Timothy Baldwin 1
- Carlos Rafael Catalan 1
- Tanmoy Chakraborty 1
- Minsuk Chang 1
- Nancy Chen 1
- Charibeth Ko Cheng 1
- Soo Kai Chie 1
- Sooyung Choi 1
- Yoonjung Choi 1
- YunSeok Choi 1
- Lheane Marie Dizon 1
- Soojong Do 1
- Bich Ngoc Doan 1
- Thong T. Doan 1
- Pham Minh Duc 1
- Sugyeong Eo 1
- Maria Regina Justina Estuar 1
- Akane Fukushige 1
- Geyang Guo 1
- Sunkyung Han 1
- Eduard Hovy 1
- Koji Inoue 1
- DongJin Jeong 1
- Seohyeong Jeong 1
- Dongjun Kang 1
- Kazushi Kato 1
- Tatsuya Kawahara 1
- Zohaib Khan 1
- Dongjun Kim 1
- Gyuwan Kim 1
- Soyeon Kim 1
- Suin Kim 1
- Young Jin Kim 1
- YoungBin Kim 1
- Donghyeok Koh 1
- Neel Kothari 1
- Dong-Ho Lee 1
- Jee-Hyong Lee 1
- Jongwuk Lee 1
- Roy Ka-Wei Lee 1
- Seungyoon Lee 1
- Yeon-Soo Lim 1
- Zhengyuan Liu 1
- Sarah Masud 1
- Jann Railey Montalan 1
- Max Müller-Eberstein 1
- CheolWon Na 1
- Tarek Naous 1
- Dongwon Noh 1
- Zi Haur Pang 1
- Cheoneum Park 1
- Nahyeon Park 1
- Sungjoon Park 1
- Tanish Patwa 1
- Jia Wang Peh 1
- Dylan Raharja 1
- Chris Reed 1
- Aina Azyyati Binti Mohamad Rezal 1
- Yonghyun Ryu 1
- Anagha Savit 1
- Gaeun Seo 1
- Jaehyung Seo 1
- Wafi Shamdi 1
- Sahajpreet Singh 1
- Liew Yu Siong 1
- Eunbeen Son 1
- Bryan Chen Zhengyu Tan 1
- Rui Yang Tan 1
- Trung Thanh Tran 1
- Lew Yan Yan Vanessa 1
- Nadya Yuki Wangsajaya 1
- Zheng Weihua 1
- Steven Euijong Whang 1
- Huadan Wu 1
- Xing Xie 1
- Xing Xie 1
- Mengyu Ye 1
- Taemin Yeom 1
- Hokeun Yoon 1
- Seunghyun Yoon 1