Lin Jingxia
Author directory2026
RegioZH: Unveiling Comprehension Disparities across Regional Chinese Varieties in LLMs
Weihua Zheng | Wu Kui | Thong T. Doan | Lin Jingxia | AiTi Aw | Nancy F. Chen | Roy Ka-Wei Lee
Findings of the Association for Computational Linguistics: EMNLP 2026
Weihua Zheng | Wu Kui | Thong T. Doan | Lin Jingxia | AiTi Aw | Nancy F. Chen | Roy Ka-Wei Lee
Findings of the Association for Computational Linguistics: EMNLP 2026
Current evaluations often treat Chinese varieties as monolithic, obscuring failures for regional users. We introduce RegioZH, a 2,607-item multiple-choice benchmark for regional lexical reasoning across Mainland China, Taiwan, Hong Kong, Singapore, and Malaysia. Built by region-specific annotators and expert-validated, it targets localized idioms, slang, code-mixing, politeness conventions, and sociocultural expressions. Zero-shot results on eleven LLMs show large disparities, including a 29.4-point average accuracy gap between Taiwan and Singapore. Persona and structured chain-of-thought prompts for five models yield selective gains, suggesting that some errors stem from retrieval or framing rather than only missing knowledge. Representation, attention, calibration, and surface-feature analyses show a dissociation between marker attention and semantic grounding, overconfident forced-choice errors, and localized lexical density as a consistent difficulty predictor. Qualitative analysis finds terminology misunderstandings, sociocultural inference gaps, and cross-regional semantic divergence. RegioZH demonstrates that apparent Chinese proficiency can mask uneven understanding across regional varieties, motivating intra-language robustness evaluation alongside multilingual generalization.