Kazushi Kato
Author directory2026
CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia
Bryan Chen Zhengyu Tan | Weihua Zheng | Thong T. Doan | Bich Ngoc Doan | Jia Wang Peh | Xiaoyuan Yi | Jing Yao | Xing Xie | Nancy F. Chen | Zhengyuan Liu | JinYeong Bak | Wafi Shamdi | Soo Kai Chie | Liew Yu Siong | Aina Azyyati Binti Mohamad Rezal | Lew Yan Yan Vanessa | Huadan Wu | Dylan Raharja | Nadya Yuki Wangsajaya | Akane Fukushige | Kazushi Kato | Koji Inoue | Tatsuya Kawahara | Jaehyung Seo | Dongjun Kim | Seungyoon Lee | Zi Haur Pang | Rui Yang Tan | Charibeth Ko Cheng | Maria Regina Justina Estuar | Jann Railey Montalan | Pham Minh Duc | Roy Ka-Wei Lee
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Bryan Chen Zhengyu Tan | Weihua Zheng | Thong T. Doan | Bich Ngoc Doan | Jia Wang Peh | Xiaoyuan Yi | Jing Yao | Xing Xie | Nancy F. Chen | Zhengyuan Liu | JinYeong Bak | Wafi Shamdi | Soo Kai Chie | Liew Yu Siong | Aina Azyyati Binti Mohamad Rezal | Lew Yan Yan Vanessa | Huadan Wu | Dylan Raharja | Nadya Yuki Wangsajaya | Akane Fukushige | Kazushi Kato | Koji Inoue | Tatsuya Kawahara | Jaehyung Seo | Dongjun Kim | Seungyoon Lee | Zi Haur Pang | Rui Yang Tan | Charibeth Ko Cheng | Maria Regina Justina Estuar | Jann Railey Montalan | Pham Minh Duc | Roy Ka-Wei Lee
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common use case: users seeking practical help over multiple turns in culturally grounded scenarios. We introduce CultureConverse, a scalable, multilingual simulation and evaluation harness for culturally grounded assistant dialogue that covers 10 East and Southeast Asian regions, 58 subgroup identities, and 7 domains. Each simulated and evaluated episode produces a scored interaction where the assistant assists the user and infers cultural constraints from partial information. The resulting CultureConverse-DS dataset contains 14,610 benchmark (evaluation) episodes and 274,295 oracle-guided (gold-mode) dialogues. In our benchmark evaluation of 18 models, GPT-5 mini achieves the highest assistance quality. Human annotation experiments suggest that our evaluation framework is a sufficient proxy for human judgment. Performance gains from fine-tuning on 27,860 high-quality CultureConverse-DS samples improve in-domain assistance and transfer out-of-domain to cultural MCQ and safety classification benchmarks. We release the harness, both splits, and judge prompts to support interactive evaluation of cultural competency.
Search
Fix author
Co-authors
- JinYeong Bak 1
- Nancy Chen 1
- Charibeth Ko Cheng 1
- Soo Kai Chie 1
- Bich Ngoc Doan 1
- Thong T. Doan 1
- Pham Minh Duc 1
- Maria Regina Justina Estuar 1
- Akane Fukushige 1
- Koji Inoue 1
- Tatsuya Kawahara 1
- Dongjun Kim 1
- Roy Ka-Wei Lee 1
- Seungyoon Lee 1
- Zhengyuan Liu 1
- Jann Railey Montalan 1
- Zi Haur Pang 1
- Jia Wang Peh 1
- Dylan Raharja 1
- Aina Azyyati Binti Mohamad Rezal 1
- Jaehyung Seo 1
- Wafi Shamdi 1
- Liew Yu Siong 1
- Bryan Chen Zhengyu Tan 1
- Rui Yang Tan 1
- Lew Yan Yan Vanessa 1
- Nadya Yuki Wangsajaya 1
- Zheng Weihua 1
- Huadan Wu 1
- Xing Xie 1
- Jing Yao 1
- Xiaoyuan Yi 1