From Errors to Precision: A Taxonomy-Guided Approach for LLM Flood Cause Extraction

Jane Arleth dela Cruz, Iris Hendrickx, Martha Larson


Abstract
Large language models (LLMs) offer disaster risk management (DRM) practitioners a promising “plug-and-play” solution for information extraction on online media data. However, they can struggle with the strict precision required for identifying semantically and structurally complex disaster causes. We evaluated how off-the shelf LLMs extract causal information from flood-related texts and found a strong recall bias. To diagnose these errors, our primary contribution is a comprehensive, human-annotated qualitative error taxonomy using inductive and deductive coding. We identified specific linguistic traps, i.e., impact descriptions, background context, and subjective opinions, trigger over-extraction, indicating models have a patterned failure. As an exploratory second step, we apply this taxonomy to design an ablation study of prompt-based interventions using definitions and exclusions. Our results show that our approach successfully shifts models along the precision–recall trade-off, i.e., curbing over-extraction, improving precision and domain-specific F0.5-score for recall-biased models. We demonstrate that by leveraging qualitative error analysis, domain experts can not only characterize and refine off-the-shelf LLMs, but also navigate the tradeoffs of prompting.
Anthology ID:
2026.wnut-1.8
Volume:
Proceedings of the 11th Workshop on Natural User-generated Text
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
JinYeong Bak, Rob van der Goot, Hyeju Jang, Weerayut Buaphet, Alan Ramponi, Wei Xu, Alan Ritter
Venues:
WNUT | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
101–116
Language:
URL:
https://aclanthology.org/2026.wnut-1.8/
DOI:
Bibkey:
Cite (ACL):
Jane Arleth dela Cruz, Iris Hendrickx, and Martha Larson. 2026. From Errors to Precision: A Taxonomy-Guided Approach for LLM Flood Cause Extraction. In Proceedings of the 11th Workshop on Natural User-generated Text, pages 101–116, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
From Errors to Precision: A Taxonomy-Guided Approach for LLM Flood Cause Extraction (dela Cruz et al., WNUT 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.wnut-1.8.pdf