AquaCrisis: A Multilingual Benchmark for Classifying Water-Utility Public Communication

Vishal Kumar, Mourad Oussalah


Abstract
Water utilities communicate both routine service information and time-sensitive public-risk guidance, yet multilingual NLP benchmarks for institution-authored infrastructure communication remain limited. We introduce AquaCrisis, a corpus of 20,000 text-bearing Facebook posts from 12 European water utilities, with a 1,500-post manually annotated gold standard. We formulate Task A as classification of a post’s main water-utility topic and Task B as classification of its primary communicative function, together with a coarser grouped Task B.1. We compare rule-based and supervised classifiers with four instruction-tuned LLMs on native-language and English-translated inputs. Supervised approaches achieve the highest point estimates on the fine-grained tasks: translated TF-IDF with logistic regression performs best on Task A, while translated multilingual sentence embeddings perform best on Task B. GPT-4o-mini achieves the highest point estimate on grouped Task B.1. Paired bootstrap analysis shows that translation effects are model- and setting-dependent: gains are clearest for the lexical TF-IDF classifier and selected Gemma 4 and GPT-4.1-mini conditions, while multilingual sentence embeddings, GPT-4o-mini, and Qwen 3.5 are comparatively stable across input representations. AquaCrisis provides a benchmark for multilingual water-utility communication spanning routine, educational, operational, and incident-related messaging, supporting communication monitoring, operational triage, and cross-lingual evaluation.
Anthology ID:
2026.climatenlp-1.18
Volume:
Proceedings of the 3rd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2026)
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Tobias Schimanski, Aida Usmanova, Gaku Morio, Jingwei Ni, Kalyan Dutia, Seyed Saeid Vaghefi, Dominik Stammbach, Markus Leippold
Venues:
ClimateNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
246–281
Language:
URL:
https://aclanthology.org/2026.climatenlp-1.18/
DOI:
Bibkey:
Cite (ACL):
Vishal Kumar and Mourad Oussalah. 2026. AquaCrisis: A Multilingual Benchmark for Classifying Water-Utility Public Communication. In Proceedings of the 3rd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2026), pages 246–281, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
AquaCrisis: A Multilingual Benchmark for Classifying Water-Utility Public Communication (Kumar & Oussalah, ClimateNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.climatenlp-1.18.pdf