What Do You See in this Patient? Behavioral Testing of Clinical NLP Models

Betty Van Aken, Sebastian Herrmann, Alexander Löser


Abstract
Decision support systems based on clinical notes have the potential to improve patient care by pointing doctors towards overseen risks. Predicting a patient’s outcome is an essential part of such systems, for which the use of deep neural networks has shown promising results. However, the patterns learned by these networks are mostly opaque and previous work revealed both reproduction of systemic biases and unexpected behavior for out-of-distribution patients. For application in clinical practice it is crucial to be aware of such behavior. We thus introduce a testing framework that evaluates clinical models regarding certain changes in the input. The framework helps to understand learned patterns and their influence on model decisions. In this work, we apply it to analyse the change in behavior with regard to the patient characteristics gender, age and ethnicity. Our evaluation of three current clinical NLP models demonstrates the concrete effects of these characteristics on the models’ decisions. They show that model behavior varies drastically even when fine-tuned on the same data with similar AUROC score. These results exemplify the need for a broader communication of model behavior in the clinical domain.
Anthology ID:
2022.clinicalnlp-1.7
Volume:
Proceedings of the 4th Clinical Natural Language Processing Workshop
Month:
July
Year:
2022
Address:
Seattle, WA
Editors:
Tristan Naumann, Steven Bethard, Kirk Roberts, Anna Rumshisky
Venue:
ClinicalNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
63–73
Language:
URL:
https://aclanthology.org/2022.clinicalnlp-1.7
DOI:
10.18653/v1/2022.clinicalnlp-1.7
Bibkey:
Cite (ACL):
Betty Van Aken, Sebastian Herrmann, and Alexander Löser. 2022. What Do You See in this Patient? Behavioral Testing of Clinical NLP Models. In Proceedings of the 4th Clinical Natural Language Processing Workshop, pages 63–73, Seattle, WA. Association for Computational Linguistics.
Cite (Informal):
What Do You See in this Patient? Behavioral Testing of Clinical NLP Models (Van Aken et al., ClinicalNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.clinicalnlp-1.7.pdf
Video:
 https://aclanthology.org/2022.clinicalnlp-1.7.mp4
Code
 bvanaken/clinical-behavioral-testing
Data
Clinical Admission Notes from MIMIC-IIIMIMIC-III