Clinical Flair: A Pre-Trained Language Model for Spanish Clinical Natural Language Processing

Matías Rojas, Jocelyn Dunstan, Fabián Villena


Abstract
Word embeddings have been widely used in Natural Language Processing (NLP) tasks. Although these representations can capture the semantic information of words, they cannot learn the sequence-level semantics. This problem can be handled using contextual word embeddings derived from pre-trained language models, which have contributed to significant improvements in several NLP tasks. Further improvements are achieved when pre-training these models on domain-specific corpora. In this paper, we introduce Clinical Flair, a domain-specific language model trained on Spanish clinical narratives. To validate the quality of the contextual representations retrieved from our model, we tested them on four named entity recognition datasets belonging to the clinical and biomedical domains. Our experiments confirm that incorporating domain-specific embeddings into classical sequence labeling architectures improves model performance dramatically compared to general-domain embeddings, demonstrating the importance of having these resources available.
Anthology ID:
2022.clinicalnlp-1.9
Volume:
Proceedings of the 4th Clinical Natural Language Processing Workshop
Month:
July
Year:
2022
Address:
Seattle, WA
Editors:
Tristan Naumann, Steven Bethard, Kirk Roberts, Anna Rumshisky
Venue:
ClinicalNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
87–92
Language:
URL:
https://aclanthology.org/2022.clinicalnlp-1.9
DOI:
10.18653/v1/2022.clinicalnlp-1.9
Bibkey:
Cite (ACL):
Matías Rojas, Jocelyn Dunstan, and Fabián Villena. 2022. Clinical Flair: A Pre-Trained Language Model for Spanish Clinical Natural Language Processing. In Proceedings of the 4th Clinical Natural Language Processing Workshop, pages 87–92, Seattle, WA. Association for Computational Linguistics.
Cite (Informal):
Clinical Flair: A Pre-Trained Language Model for Spanish Clinical Natural Language Processing (Rojas et al., ClinicalNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.clinicalnlp-1.9.pdf
Video:
 https://aclanthology.org/2022.clinicalnlp-1.9.mp4
Code
 plncmm/spanish-clinical-flair
Data
Chilean Waiting List