End-to-end clinical temporal information extraction with multi-head attention

Timothy Miller, Steven Bethard, Dmitriy Dligach, Guergana Savova


Abstract
Understanding temporal relationships in text from electronic health records can be valuable for many important downstream clinical applications. Since Clinical TempEval 2017, there has been little work on end-to-end systems for temporal relation extraction, with most work focused on the setting where gold standard events and time expressions are given. In this work, we make use of a novel multi-headed attention mechanism on top of a pre-trained transformer encoder to allow the learning process to attend to multiple aspects of the contextualized embeddings. Our system achieves state of the art results on the THYME corpus by a wide margin, in both the in-domain and cross-domain settings.
Anthology ID:
2023.bionlp-1.28
Volume:
The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Dina Demner-fushman, Sophia Ananiadou, Kevin Cohen
Venue:
BioNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
313–319
Language:
URL:
https://aclanthology.org/2023.bionlp-1.28
DOI:
10.18653/v1/2023.bionlp-1.28
Bibkey:
Cite (ACL):
Timothy Miller, Steven Bethard, Dmitriy Dligach, and Guergana Savova. 2023. End-to-end clinical temporal information extraction with multi-head attention. In The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, pages 313–319, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
End-to-end clinical temporal information extraction with multi-head attention (Miller et al., BioNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.bionlp-1.28.pdf