<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="nawzad-2026-kummlu">
    <titleInfo>
        <title>KuMMLU: A Multitask Language Understanding Benchmark for Central Kurdish</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Akam</namePart>
        <namePart type="family">Nawzad</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 10th Widening NLP Workshop</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Chen</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Emily</namePart>
            <namePart type="family">Allaway</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Meryem</namePart>
            <namePart type="family">M’hamdi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Peerat</namePart>
            <namePart type="family">Limkonchotiwat</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Richard</namePart>
            <namePart type="given">He</namePart>
            <namePart type="family">Bai</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Santosh</namePart>
            <namePart type="family">T.Y.S.S.</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Surendrabikram</namePart>
            <namePart type="family">Thapa</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yinqiao</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-472-9</identifier>
    </relatedItem>
    <abstract>Language models are now tested across many languages, but Kurdish is mostly missing from these efforts. We present KuMMLU, a multiple-choice benchmark for Central Kurdish (Sorani) built from the national examinations of the Kurdistan Region of Iraq. It contains 3,272 questions across 18 subjects: 2,256 native examination questions, with answers taken from the official keys published by the Ministry of Education, and 1,016 questions from six subjects translated from English MMLU. We evaluate 32 language models, including proprietary systems, open-weight models, and multilingual models built for wide language coverage. The best models reach about 88% accuracy, while the five multilingual baselines score between 15.8% and 37.2%, at most modestly above the 25% chance level and well below the strongest general-purpose systems; this includes the one model whose published language list contains Kurdish. Every model scores higher on the translated subset than on the native questions. We examine this gap in light of subject difficulty and possible overlap with training data, and release the benchmark to support Kurdish language technology.</abstract>
    <identifier type="citekey">nawzad-2026-kummlu</identifier>
    <location>
        <url>https://aclanthology.org/2026.winlp-main.15/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>173</start>
            <end>180</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T KuMMLU: A Multitask Language Understanding Benchmark for Central Kurdish
%A Nawzad, Akam
%Y Zhang, Chen
%Y Allaway, Emily
%Y M’hamdi, Meryem
%Y Limkonchotiwat, Peerat
%Y Bai, Richard He
%Y T.Y.S.S., Santosh
%Y Thapa, Surendrabikram
%Y Li, Yinqiao
%S Proceedings of the 10th Widening NLP Workshop
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-472-9
%F nawzad-2026-kummlu
%X Language models are now tested across many languages, but Kurdish is mostly missing from these efforts. We present KuMMLU, a multiple-choice benchmark for Central Kurdish (Sorani) built from the national examinations of the Kurdistan Region of Iraq. It contains 3,272 questions across 18 subjects: 2,256 native examination questions, with answers taken from the official keys published by the Ministry of Education, and 1,016 questions from six subjects translated from English MMLU. We evaluate 32 language models, including proprietary systems, open-weight models, and multilingual models built for wide language coverage. The best models reach about 88% accuracy, while the five multilingual baselines score between 15.8% and 37.2%, at most modestly above the 25% chance level and well below the strongest general-purpose systems; this includes the one model whose published language list contains Kurdish. Every model scores higher on the translated subset than on the native questions. We examine this gap in light of subject difficulty and possible overlap with training data, and release the benchmark to support Kurdish language technology.
%U https://aclanthology.org/2026.winlp-main.15/
%P 173-180

Markdown (Informal)

[KuMMLU: A Multitask Language Understanding Benchmark for Central Kurdish](https://aclanthology.org/2026.winlp-main.15/) (Nawzad, WiNLP 2026)

• KuMMLU: A Multitask Language Understanding Benchmark for Central Kurdish (Nawzad, WiNLP 2026)

ACL

• Akam Nawzad. 2026. KuMMLU: A Multitask Language Understanding Benchmark for Central Kurdish. In Proceedings of the 10th Widening NLP Workshop, pages 173–180, Budapest, Hungary. Association for Computational Linguistics.