Corporate climate transition plans detail how companies intend to reduce their carbon footprint, but their unstructured and often incomplete nature limits assessments of whether disclosed actions are sufficiently concrete, quantified, and consistent with stated emission-reduction objectives. We analyze transition plans from CDP by decomposing them into the key information needed for such assessments: disclosed mitigation measures, their implementation status, quantitative attributes, action type, specificity, and organizational perimeter. Large language models achieve moderate to strong performance across these tasks (65-91% F1), enabling large-scale diagnostic analysis of transition-plan disclosures. Our results show that the main bottleneck lies in the disclosures themselves. Specifically, 80% of transition plans contain at most two quantified planned measures, and nearly 47% of proposed actions are both generic and lack a specified organizational perimeter. Firms also tend to favor high-level targets, preparatory actions, and energy-related measures over more structural business transformations. Overall, our findings show that current transition-plan reporting often lacks the granularity needed to assess whether disclosed actions are consistent with stated emission-reduction targets.

Significant efforts have been made in the NLP community to facilitate the automatic analysis of climate-related corpora by tasks such as climate-related topic detection, climate risk classification, question answering over climate topics, and many more. In this work, we perform a reproducibility study on 8 tasks and 29 datasets, testing 6 models. We find that many tasks rely heavily on surface-level keyword patterns rather than deeper semantic or contextual understanding. Moreover, we find that 96% of the datasets contain annotation issues, with 16.6% of the sampled wrong predictions of a zero-shot classifier being actually clear annotation mistakes, and 38.8% being ambiguous examples.These results call into question the reliability of current benchmarks to meaningfully compare models and highlight the need for improved annotation practices. We conclude by outlining actionable recommendations to enhance dataset quality and evaluation robustness.

We introduce MAFALDA, a benchmark for fallacy classification that merges and unites previous fallacy datasets. It comes with a taxonomy that aligns, refines, and unifies existing classifications of fallacies. We further provide a manual annotation of a part of the dataset together with manual explanations for each annotation. We propose a new annotation scheme tailored for subjective NLP tasks, and a new evaluation method designed to handle subjectivity. We then evaluate several language models under a zero-shot learning setting and human performances on MAFALDA to assess their capability to detect and classify fallacies.