Prateek Chanda
2024
Few shot chain-of-thought driven reasoning to prompt LLMs for open-ended medical question answering
Saeel Nachane
|
Ojas Gramopadhye
|
Prateek Chanda
|
Ganesh Ramakrishnan
|
Kshitij Jadhav
|
Yatin Nandwani
|
Dinesh Raghu
|
Sachindra Joshi
Findings of the Association for Computational Linguistics: EMNLP 2024
In this paper, we propose a modified version of the MedQA-USMLE dataset, named MEDQA-OPEN, which contains open-ended medical questions without options to mimic clinical scenarios, along with clinician-approved reasoned answers. Additionally, we implement a prompt driven by Chain of Thought (CoT) reasoning, CLINICR, to mirror the prospective process of incremental reasoning, reaching a correct response to medical questions. We empirically demonstrate how CLINICR outperforms the state-of-the-art 5-shot CoT-based prompt (Liévin et al., 2022). We also present an approach that mirrors real-life clinical practice by first exploring multiple differential diagnoses through MCQ-CLINICR and subsequently narrowing down to a final diagnosis using MCQ-ELIMINATIVE. Finally, emphasizing the importance of response verification in medical settings, we utilize a reward model mechanism, replacing the elimination process performed by MCQ-ELIMINATIVE.
Search
Co-authors
- Saeel Nachane 1
- Ojas Gramopadhye 1
- Ganesh Ramakrishnan 1
- Kshitij Jadhav 1
- Yatin Nandwani 1
- show all...