CA-ClimateNews: A Human-Annotated Multilingual Benchmark for Climate-Relevance Classification of Central Asian News

Adilet Metinov, Zarina Uvalieva, Bektemir Kumarbai Uulu, Nurtilek Alibekov, Tynchtykbek Tashbaltaev


Abstract
Climate change is acute in Central Asia (glacier loss, the Aral Sea, and drought exposure), yet there are no NLP resources for climate discourse in the region’s low-resource Turkic languages. We present CA-ClimateNews, the first human-annotated multilingual dataset for climate-relevance classification of Central Asian news, covering Kyrgyz, Kazakh, Uzbek, and (regionally grounded) Russian. The corpus comprises 3,140 news items labeled at two levels (binary climate relevance and an 8-way topic category) by two annotators per language, with a 100-item blind double-annotation overlap adjudicated to gold. Inter-annotator agreement is almost perfect (Cohen’s 𝜅 0.86–0.98 on relevance). Because we used transparent LLM-assisted pre-labeling, we additionally run a between-subjects experiment showing that displaying model pre-labels induces anchoring: an independent group shown the pre-label overrode it 3–7× less than a blind group. Our released labels are the blind version. We report baselines: fine-tuned multilingual encoders reach 88–96 relevance macro-F1 on the balanced benchmark; a Russian-only model loses up to ∼7 points transferring zero-shot to the Turkic languages even at matched training size (least, ∼2, for Kazakh), and a related Turkic source is no better, isolating a cross-lingual gap that in-language supervision closes. The dataset, guidelines, and code are publicly released.
Anthology ID:
2026.climatenlp-1.2
Volume:
Proceedings of the 3rd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2026)
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Tobias Schimanski, Aida Usmanova, Gaku Morio, Jingwei Ni, Kalyan Dutia, Seyed Saeid Vaghefi, Dominik Stammbach, Markus Leippold
Venues:
ClimateNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
20–28
Language:
URL:
https://aclanthology.org/2026.climatenlp-1.2/
DOI:
Bibkey:
Cite (ACL):
Adilet Metinov, Zarina Uvalieva, Bektemir Kumarbai Uulu, Nurtilek Alibekov, and Tynchtykbek Tashbaltaev. 2026. CA-ClimateNews: A Human-Annotated Multilingual Benchmark for Climate-Relevance Classification of Central Asian News. In Proceedings of the 3rd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2026), pages 20–28, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
CA-ClimateNews: A Human-Annotated Multilingual Benchmark for Climate-Relevance Classification of Central Asian News (Metinov et al., ClimateNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.climatenlp-1.2.pdf