Luca Giordano


2024

pdf bib
Large Language Models as Drug Information Providers for Patients
Luca Giordano | Maria Pia di Buono
Proceedings of the First Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC-COLING 2024

Recently, a significant interest has arisen about the application of Large Language Models (LLMs) in medical settings to enhance various aspects of healthcare. Particularly, the application of such models to improve knowledge access for both clinicians and patients seems very promising but still far from perfect. In this paper, we present a preliminary evaluation of LLMs as drug information providers to support patients in drug administration. We focus on posology, namely dosage quantity and prescription, contraindications and adverse drug reactions and run an experiment on the Italian language to assess both the trustworthiness of the outputs and their readability. The results show that different types of errors affect the LLM answers. In some cases, the model does not recognize the drug name, due to the presence of synonymous words, or it provides untrustworthy information, caused by intrinsic hallucinations. Overall, the complexity of the language is lower and this could contribute to make medical information more accessible to lay people.

pdf bib
DIMMI - Drug InforMation Mining in Italian: A CALAMITA Challenge
Raffaele Manna | Maria Pia Di Buono | Luca Giordano
Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024)

Patients’ knowledge about drugs and medications is crucial as it allows them to administer them safely. This knowledgefrequently comes from written prescriptions, patient information leaflets (PILs), or from reading drug Web pages. DIMMI(Drug InforMation Mining in Italian) is a challenge aiming at evaluating the proficiency of Large Language Models in extractingdrug-specific information from PILs. The challenge seeks to advance the understanding of effectiveness in processing complexmedical information in Italian, and to enhance drug information extraction and pharmacovigilance efforts. Participants areprovided with a dataset of 600 Italian PILs and the objective is to develop models capable of accurately answering specificquestions related to drug dosage, usage, side effects, drug-drug interactions. The challenge should be approached as aninformation extraction task through a zero-shot mode, purely based on the model pre-existing knowledge and understandingor through in-context learning (Retrieval-Augmented Generation (RAG) or few-shot mode). The answers generated by themodels will be compared against the gold standard (GS), created to establish a reliable, accurate, and a comprehensive setof answers against which participant submissions can be evaluated. For each drug and each information category, the GScontains the correct information extracted from the leaflets through a manual annotation.

2023

pdf bib
Assessing Italian News Reliability in the Health Domain through Text Analysis of Headlines
Luca Giordano | Maria Pia Di Buono
Proceedings of the 4th Conference on Language, Data and Knowledge