Ensemble-based Fine-Tuning Strategy for Temporal Relation Extraction from the Clinical Narrative

Lijing Wang, Timothy Miller, Steven Bethard, Guergana Savova


Abstract
In this paper, we investigate ensemble methods for fine-tuning transformer-based pretrained models for clinical natural language processing tasks, specifically temporal relation extraction from the clinical narrative. Our experimental results on the THYME data show that ensembling as a fine-tuning strategy can further boost model performance over single learners optimized for hyperparameters. Dynamic snapshot ensembling is particularly beneficial as it fine-tunes a wide array of parameters and results in a 2.8% absolute improvement in F1 over the base single learner.
Anthology ID:
2022.clinicalnlp-1.11
Volume:
Proceedings of the 4th Clinical Natural Language Processing Workshop
Month:
July
Year:
2022
Address:
Seattle, WA
Editors:
Tristan Naumann, Steven Bethard, Kirk Roberts, Anna Rumshisky
Venue:
ClinicalNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
103–108
Language:
URL:
https://aclanthology.org/2022.clinicalnlp-1.11
DOI:
10.18653/v1/2022.clinicalnlp-1.11
Bibkey:
Cite (ACL):
Lijing Wang, Timothy Miller, Steven Bethard, and Guergana Savova. 2022. Ensemble-based Fine-Tuning Strategy for Temporal Relation Extraction from the Clinical Narrative. In Proceedings of the 4th Clinical Natural Language Processing Workshop, pages 103–108, Seattle, WA. Association for Computational Linguistics.
Cite (Informal):
Ensemble-based Fine-Tuning Strategy for Temporal Relation Extraction from the Clinical Narrative (Wang et al., ClinicalNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.clinicalnlp-1.11.pdf