Joint Learning with Pre-trained Transformer on Named Entity Recognition and Relation Extraction Tasks for Clinical Analytics

Miao Chen, Ganhui Lan, Fang Du, Victor Lobanov


Abstract
In drug development, protocols define how clinical trials are conducted, and are therefore of paramount importance. They contain key patient-, investigator-, medication-, and study-related information, often elaborated in different sections in the protocol texts. Granular-level parsing on large quantity of existing protocols can accelerate clinical trial design and provide actionable insights into trial optimization. Here, we report our progresses in using deep learning NLP algorithms to enable automated protocol analytics. In particular, we combined a pre-trained BERT transformer model with joint-learning strategies to simultaneously identify clinically relevant entities (i.e. Named Entity Recognition) and extract the syntactic relations between these entities (i.e. Relation Extraction) from the eligibility criteria section in protocol texts. When comparing to standalone NER and RE models, our joint-learning strategy can effectively improve the performance of RE task while retaining similarly high NER performance, likely due to the synergy of optimizing toward both tasks’ objectives via shared parameters. The derived NLP model provides an end-to-end solution to convert unstructured protocol texts into structured data source, which will be embedded into a comprehensive clinical analytics workflow for downstream trial design missions such like patient population extraction, patient enrollment rate estimation, and protocol amendment prediction.
Anthology ID:
2020.clinicalnlp-1.26
Volume:
Proceedings of the 3rd Clinical Natural Language Processing Workshop
Month:
November
Year:
2020
Address:
Online
Venues:
ClinicalNLP | EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
234–242
Language:
URL:
https://aclanthology.org/2020.clinicalnlp-1.26
DOI:
10.18653/v1/2020.clinicalnlp-1.26
Bibkey:
Cite (ACL):
Miao Chen, Ganhui Lan, Fang Du, and Victor Lobanov. 2020. Joint Learning with Pre-trained Transformer on Named Entity Recognition and Relation Extraction Tasks for Clinical Analytics. In Proceedings of the 3rd Clinical Natural Language Processing Workshop, pages 234–242, Online. Association for Computational Linguistics.
Cite (Informal):
Joint Learning with Pre-trained Transformer on Named Entity Recognition and Relation Extraction Tasks for Clinical Analytics (Chen et al., ClinicalNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.clinicalnlp-1.26.pdf
Video:
 https://slideslive.com/38939837