SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding

Peerawat Chomphooyod, Yosephine Susanto, Jian Gang Ngui, Attapol Rutherford, Alham Fikri Aji, Sarana Nutanong, Can Udomcharoenchaikit, Peerat Limkonchotiwat


Abstract
Frontier LLMs perform well in Western contexts, but remain poorly tested on underrepresented cultures such as those in Southeast Asia (SEA). Existing NLI benchmarks are largely Western-centric, translation-derived, or monolingual, limiting their ability to measure culturally grounded reasoning. We introduce SEA-NLI, a native, culturally grounded NLI benchmark covering eight SEA countries in English and native regional languages, verified by native speakers. Across 17 encoder and decoder models, we observe a low performance from all models, especially for knowledge-intensive categories such as Languages and Science and Technology. Our analysis shows that failure cases mainly stem from missing SEA cultural knowledge: SEA-adapted models and culture-aware prompting improve performance, while CoT prompting offers limited gains.
Anthology ID:
2026.aacl-long.94
Volume:
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venues:
AACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2038–2062
Language:
URL:
https://aclanthology.org/2026.aacl-long.94/
DOI:
Bibkey:
Cite (ACL):
Peerawat Chomphooyod, Yosephine Susanto, Jian Gang Ngui, Attapol Rutherford, Alham Fikri Aji, Sarana Nutanong, Can Udomcharoenchaikit, and Peerat Limkonchotiwat. 2026. SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2038–2062, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding (Chomphooyod et al., AACL-IJCNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aacl-long.94.pdf