Doctor XAvIer: Explainable Diagnosis on Physician-Patient Dialogues and XAI Evaluation

Hillary Ngai, Frank Rudzicz


Abstract
We introduce Doctor XAvIer — a BERT-based diagnostic system that extracts relevant clinical data from transcribed patient-doctor dialogues and explains predictions using feature attribution methods. We present a novel performance plot and evaluation metric for feature attribution methods — Feature Attribution Dropping (FAD) curve and its Normalized Area Under the Curve (N-AUC). FAD curve analysis shows that integrated gradients outperforms Shapley values in explaining diagnosis classification. Doctor XAvIer outperforms the baseline with 0.97 F1-score in named entity recognition and symptom pertinence classification and 0.91 F1-score in diagnosis classification.
Anthology ID:
2022.bionlp-1.33
Volume:
Proceedings of the 21st Workshop on Biomedical Language Processing
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, Junichi Tsujii
Venue:
BioNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
337–344
Language:
URL:
https://aclanthology.org/2022.bionlp-1.33
DOI:
10.18653/v1/2022.bionlp-1.33
Bibkey:
Cite (ACL):
Hillary Ngai and Frank Rudzicz. 2022. Doctor XAvIer: Explainable Diagnosis on Physician-Patient Dialogues and XAI Evaluation. In Proceedings of the 21st Workshop on Biomedical Language Processing, pages 337–344, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Doctor XAvIer: Explainable Diagnosis on Physician-Patient Dialogues and XAI Evaluation (Ngai & Rudzicz, BioNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.bionlp-1.33.pdf
Video:
 https://aclanthology.org/2022.bionlp-1.33.mp4
Code
 hillary-ngai/doctor_xavier