My Climate Advisor: An Application of NLP in Climate Adaptation for Agriculture

Vincent Nguyen, Sarvnaz Karimi, Willow Hallgren, Ashley Harkin, Mahesh Prakash


Abstract
Climate adaptation in the agricultural sector necessitates tools that equip farmers and farm advisors with relevant and trustworthy information to help increase their resiliency to climate change. We introduce My Climate Advisor, a question-answering (QA) prototype that synthesises information from different data sources, such as peer-reviewed scientific literature and high-quality, industry-relevant grey literature to generate answers, with references, to a given user’s question. Our prototype uses open-source generative models for data privacy and intellectual property protection, and retrieval augmented generation for answer generation, grounding and provenance. While there are standard evaluation metrics for QA systems, no existing evaluation framework suits our LLM-based QA application in the climate adaptation domain. We design an evaluation framework with seven metrics based on the requirements of the domain experts to judge the generated answers from 12 different LLM-based models. Our initial evaluations through a user study via domain experts show promising usability results.
Anthology ID:
2024.climatenlp-1.3
Volume:
Proceedings of the 1st Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2024)
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Dominik Stammbach, Jingwei Ni, Tobias Schimanski, Kalyan Dutia, Alok Singh, Julia Bingler, Christophe Christiaen, Neetu Kushwaha, Veruska Muccione, Saeid A. Vaghefi, Markus Leippold
Venues:
ClimateNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
27–45
Language:
URL:
https://aclanthology.org/2024.climatenlp-1.3
DOI:
10.18653/v1/2024.climatenlp-1.3
Bibkey:
Cite (ACL):
Vincent Nguyen, Sarvnaz Karimi, Willow Hallgren, Ashley Harkin, and Mahesh Prakash. 2024. My Climate Advisor: An Application of NLP in Climate Adaptation for Agriculture. In Proceedings of the 1st Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2024), pages 27–45, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
My Climate Advisor: An Application of NLP in Climate Adaptation for Agriculture (Nguyen et al., ClimateNLP-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.climatenlp-1.3.pdf