Akam Nawzad

Author directory

2026

Language models are now tested across many languages, but Kurdish is mostly missing from these efforts. We present KuMMLU, a multiple-choice benchmark for Central Kurdish (Sorani) built from the national examinations of the Kurdistan Region of Iraq. It contains 3,272 questions across 18 subjects: 2,256 native examination questions, with answers taken from the official keys published by the Ministry of Education, and 1,016 questions from six subjects translated from English MMLU. We evaluate 32 language models, including proprietary systems, open-weight models, and multilingual models built for wide language coverage. The best models reach about 88% accuracy, while the five multilingual baselines score between 15.8% and 37.2%, at most modestly above the 25% chance level and well below the strongest general-purpose systems; this includes the one model whose published language list contains Kurdish. Every model scores higher on the translated subset than on the native questions. We examine this gap in light of subject difficulty and possible overlap with training data, and release the benchmark to support Kurdish language technology.

2025

Named Entity Recognition (NER) is the information extraction task of identifying predefined named entities such as person names, location names, organization names and more. High-resource languages have made significant progress in NER tasks. However, low-resource languages such as Kurmanji Kurdish have not seen the same advancements, due to these languages having less available data online. This research aims to close this gap by developing an NER system via fine-tuning XLM-RoBERTa on a manually annotated dataset for Kurmanji. The dataset used for fine-tuning consists of 7,919 annotated sentences, which were manually annotated by three native Kurmanji speakers. The classes labeled in the dataset are Person (PER), Organization (ORG), and Location (LOC). A web-based application has also been developed using Streamlit to make the model more accessible. The model achieved an F1 score of 0.8735, precision of 0.8668, and recall of 0.8803, demonstrating the effectiveness of fine-tuning transformer-based models for NER tasks in low-resource languages. This work establishes a methodology that can be applied to other low-resource languages and Kurdish varieties.