Jinrong Ma
Author directory2026
Meeseeks-Multilingual: A Multi-Turn, Multi-Constraint Benchmark evaluating Multilingual Instruction-Following and Self-Correction
Jiaming Wang | Hefei Chen | Zhe Tang | Zehao Jin | Yanxin Zhuo | Ruicheng Liao | Jinrong Ma | Xiaoyu Li | Xuezhi Cao
Findings of the Association for Computational Linguistics: EMNLP 2026
Jiaming Wang | Hefei Chen | Zhe Tang | Zehao Jin | Yanxin Zhuo | Ruicheng Liao | Jinrong Ma | Xiaoyu Li | Xuezhi Cao
Findings of the Association for Computational Linguistics: EMNLP 2026
Large Language Models (LLMs) fail to reliably follow multi-constraint instructions across languages, and the failure rate rises when regional linguistic rules interact with general constraints. To address this challenge, we present Meeseeks-Multilingual, a multi-turn, feedback-driven benchmark that extends Meeseeks to multilingual and cross-regional evaluation. The benchmark separates general constraints (e.g., word counts, keyword rules) from regional constraints (e.g., Japanese kana requirements) so that the two failure modes can be measured independently. Meeseeks-Multilingual automatically identifies unmet constraints and provides targeted feedback to test iterative self-correction. All data are produced by trained annotators with no LLM involvement in data generation. Experiments across mainstream commercial and open-source LLMs show that even with iterative feedback, models consistently fail to meet all multilingual constraints. Aggregate model rankings stabilize by Round 5, which lets the benchmark be run at reduced cost for overall comparisons, although regional rankings settle more slowly and need more rounds. Meeseeks-Multilingual has been open-sourced on https://github.com/meituan-longcat/Meeseeks.