Characterizing the Value of Information in Medical Notes

Chao-Chun Hsu, Shantanu Karnwal, Sendhil Mullainathan, Ziad Obermeyer, Chenhao Tan


Abstract
Machine learning models depend on the quality of input data. As electronic health records are widely adopted, the amount of data in health care is growing, along with complaints about the quality of medical notes. We use two prediction tasks, readmission prediction and in-hospital mortality prediction, to characterize the value of information in medical notes. We show that as a whole, medical notes only provide additional predictive power over structured information in readmission prediction. We further propose a probing framework to select parts of notes that enable more accurate predictions than using all notes, despite that the selected information leads to a distribution shift from the training data (“all notes”). Finally, we demonstrate that models trained on the selected valuable information achieve even better predictive performance, with only 6.8%of all the tokens for readmission prediction.
Anthology ID:
2020.findings-emnlp.187
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2020
Month:
November
Year:
2020
Address:
Online
Editors:
Trevor Cohn, Yulan He, Yang Liu
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2062–2072
Language:
URL:
https://aclanthology.org/2020.findings-emnlp.187
DOI:
10.18653/v1/2020.findings-emnlp.187
Bibkey:
Cite (ACL):
Chao-Chun Hsu, Shantanu Karnwal, Sendhil Mullainathan, Ziad Obermeyer, and Chenhao Tan. 2020. Characterizing the Value of Information in Medical Notes. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2062–2072, Online. Association for Computational Linguistics.
Cite (Informal):
Characterizing the Value of Information in Medical Notes (Hsu et al., Findings 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.findings-emnlp.187.pdf
Optional supplementary material:
 2020.findings-emnlp.187.OptionalSupplementaryMaterial.zip
Video:
 https://slideslive.com/38940179
Code
 BoulderDS/value-of-medical-notes