@inproceedings{lin-etal-2026-culturall,
title = "{C}ultur{ALL}: Benchmarking Multilingual and Multicultural Competence of {LLM}s on Grounded Tasks",
author = "Lin, Peiqin and
Lyu, Chenyang and
Luo, Wenjiang and
Ye, Haotian and
Hossain, Md Mehrab and
Ma, Chunlan and
Ji, Shaoxiong and
Samih, Younes and
Zeng, Bo and
Jiang, Fan and
Cao, Yuanbin and
Duisenbek, Dilda and
Xun, Adrian Neo Sau and
Daria, Pozdniakova and
Liubou, Misevich and
Nevena, Marinkovi{\'c} and
Nguyen, Ngoc Gia Linh and
DO, Thi Khanh Linh and
Sophy, Sarakmatak and
Hu, Baotian and
Chen, Guanhua and
Tang, Gongbo and
Aji, Alham Fikri and
Wang, Longyue and
Luo, Weihua",
editor = "Zhang, Yue and
Xu, Ruifeng and
Purwarianti, Ayu and
Rong, Tong and
Suzuki, Jun",
booktitle = "Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the {A}sia-Pacific Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
month = nov,
year = "2026",
address = "Hengqin, China",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.aacl-long.188/",
pages = "4208--4224",
ISBN = "979-8-89176-439-2",
abstract = "Large language models (LLMs) are now deployed worldwide, inspiring a surge of benchmarks that measure their multilingual and multicultural abilities. However, these benchmarks prioritize generic language understanding or superficial cultural trivia, leaving the evaluation of grounded tasks{---}where models must reason within real-world, context-rich scenarios{---}largely unaddressed. To fill this gap, we present CulturALL, a comprehensive and challenging benchmark to assess LLMs' multilingual and multicultural competence on grounded tasks. CulturALL is built via a human{--}AI collaborative framework: expert annotators ensure appropriate difficulty and factual accuracy, while LLMs lighten the manual workload. By incorporating diverse sources, CulturALL ensures comprehensive scenario coverage. Each item is carefully designed to present a high level of difficulty, making CulturALL challenging. CulturALL contains 2,610 samples in 14 languages of 51 regions, distributed across 16 topics to capture the full breadth of grounded tasks. The experiments show that the best LLM achieve 44.48{\%} accuracy on CulturALL, underscoring substantial room for improvement."
}<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lin-etal-2026-culturall">
<titleInfo>
<title>CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks</title>
</titleInfo>
<name type="personal">
<namePart type="given">Peiqin</namePart>
<namePart type="family">Lin</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Chenyang</namePart>
<namePart type="family">Lyu</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Wenjiang</namePart>
<namePart type="family">Luo</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Haotian</namePart>
<namePart type="family">Ye</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Md</namePart>
<namePart type="given">Mehrab</namePart>
<namePart type="family">Hossain</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Chunlan</namePart>
<namePart type="family">Ma</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Shaoxiong</namePart>
<namePart type="family">Ji</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Younes</namePart>
<namePart type="family">Samih</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Bo</namePart>
<namePart type="family">Zeng</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Fan</namePart>
<namePart type="family">Jiang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Yuanbin</namePart>
<namePart type="family">Cao</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Dilda</namePart>
<namePart type="family">Duisenbek</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Adrian</namePart>
<namePart type="given">Neo</namePart>
<namePart type="given">Sau</namePart>
<namePart type="family">Xun</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Pozdniakova</namePart>
<namePart type="family">Daria</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Misevich</namePart>
<namePart type="family">Liubou</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Marinković</namePart>
<namePart type="family">Nevena</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Ngoc</namePart>
<namePart type="given">Gia</namePart>
<namePart type="given">Linh</namePart>
<namePart type="family">Nguyen</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Thi</namePart>
<namePart type="given">Khanh</namePart>
<namePart type="given">Linh</namePart>
<namePart type="family">DO</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Sarakmatak</namePart>
<namePart type="family">Sophy</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Baotian</namePart>
<namePart type="family">Hu</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Guanhua</namePart>
<namePart type="family">Chen</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Gongbo</namePart>
<namePart type="family">Tang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Alham</namePart>
<namePart type="given">Fikri</namePart>
<namePart type="family">Aji</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Longyue</namePart>
<namePart type="family">Wang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Weihua</namePart>
<namePart type="family">Luo</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<originInfo>
<dateIssued>2026-11</dateIssued>
</originInfo>
<typeOfResource>text</typeOfResource>
<relatedItem type="host">
<titleInfo>
<title>Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
</titleInfo>
<name type="personal">
<namePart type="given">Yue</namePart>
<namePart type="family">Zhang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Ruifeng</namePart>
<namePart type="family">Xu</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Ayu</namePart>
<namePart type="family">Purwarianti</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Tong</namePart>
<namePart type="family">Rong</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jun</namePart>
<namePart type="family">Suzuki</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<originInfo>
<publisher>Association for Computational Linguistics</publisher>
<place>
<placeTerm type="text">Hengqin, China</placeTerm>
</place>
</originInfo>
<genre authority="marcgt">conference publication</genre>
<identifier type="isbn">979-8-89176-439-2</identifier>
</relatedItem>
<abstract>Large language models (LLMs) are now deployed worldwide, inspiring a surge of benchmarks that measure their multilingual and multicultural abilities. However, these benchmarks prioritize generic language understanding or superficial cultural trivia, leaving the evaluation of grounded tasks—where models must reason within real-world, context-rich scenarios—largely unaddressed. To fill this gap, we present CulturALL, a comprehensive and challenging benchmark to assess LLMs’ multilingual and multicultural competence on grounded tasks. CulturALL is built via a human–AI collaborative framework: expert annotators ensure appropriate difficulty and factual accuracy, while LLMs lighten the manual workload. By incorporating diverse sources, CulturALL ensures comprehensive scenario coverage. Each item is carefully designed to present a high level of difficulty, making CulturALL challenging. CulturALL contains 2,610 samples in 14 languages of 51 regions, distributed across 16 topics to capture the full breadth of grounded tasks. The experiments show that the best LLM achieve 44.48% accuracy on CulturALL, underscoring substantial room for improvement.</abstract>
<identifier type="citekey">lin-etal-2026-culturall</identifier>
<location>
<url>https://aclanthology.org/2026.aacl-long.188/</url>
</location>
<part>
<date>2026-11</date>
<extent unit="page">
<start>4208</start>
<end>4224</end>
</extent>
</part>
</mods>
</modsCollection>
%0 Conference Proceedings
%T CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks
%A Lin, Peiqin
%A Lyu, Chenyang
%A Luo, Wenjiang
%A Ye, Haotian
%A Hossain, Md Mehrab
%A Ma, Chunlan
%A Ji, Shaoxiong
%A Samih, Younes
%A Zeng, Bo
%A Jiang, Fan
%A Cao, Yuanbin
%A Duisenbek, Dilda
%A Xun, Adrian Neo Sau
%A Daria, Pozdniakova
%A Liubou, Misevich
%A Nevena, Marinković
%A Nguyen, Ngoc Gia Linh
%A DO, Thi Khanh Linh
%A Sophy, Sarakmatak
%A Hu, Baotian
%A Chen, Guanhua
%A Tang, Gongbo
%A Aji, Alham Fikri
%A Wang, Longyue
%A Luo, Weihua
%Y Zhang, Yue
%Y Xu, Ruifeng
%Y Purwarianti, Ayu
%Y Rong, Tong
%Y Suzuki, Jun
%S Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2026
%8 November
%I Association for Computational Linguistics
%C Hengqin, China
%@ 979-8-89176-439-2
%F lin-etal-2026-culturall
%X Large language models (LLMs) are now deployed worldwide, inspiring a surge of benchmarks that measure their multilingual and multicultural abilities. However, these benchmarks prioritize generic language understanding or superficial cultural trivia, leaving the evaluation of grounded tasks—where models must reason within real-world, context-rich scenarios—largely unaddressed. To fill this gap, we present CulturALL, a comprehensive and challenging benchmark to assess LLMs’ multilingual and multicultural competence on grounded tasks. CulturALL is built via a human–AI collaborative framework: expert annotators ensure appropriate difficulty and factual accuracy, while LLMs lighten the manual workload. By incorporating diverse sources, CulturALL ensures comprehensive scenario coverage. Each item is carefully designed to present a high level of difficulty, making CulturALL challenging. CulturALL contains 2,610 samples in 14 languages of 51 regions, distributed across 16 topics to capture the full breadth of grounded tasks. The experiments show that the best LLM achieve 44.48% accuracy on CulturALL, underscoring substantial room for improvement.
%U https://aclanthology.org/2026.aacl-long.188/
%P 4208-4224
Markdown (Informal)
[CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks](https://aclanthology.org/2026.aacl-long.188/) (Lin et al., AACL-IJCNLP 2026)
ACL
- Peiqin Lin, Chenyang Lyu, Wenjiang Luo, Haotian Ye, Md Mehrab Hossain, Chunlan Ma, Shaoxiong Ji, Younes Samih, Bo Zeng, Fan Jiang, Yuanbin Cao, Dilda Duisenbek, Adrian Neo Sau Xun, Pozdniakova Daria, Misevich Liubou, Marinković Nevena, Ngoc Gia Linh Nguyen, Thi Khanh Linh DO, Sarakmatak Sophy, Baotian Hu, Guanhua Chen, Gongbo Tang, Alham Fikri Aji, Longyue Wang, and Weihua Luo. 2026. CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4208–4224, Hengqin, China. Association for Computational Linguistics.