Scalable Few-Shot Learning of Robust Biomedical Name Representations

Pieter Fivez, Simon Suster, Walter Daelemans


Abstract
Recent research on robust representations of biomedical names has focused on modeling large amounts of fine-grained conceptual distinctions using complex neural encoders. In this paper, we explore the opposite paradigm: training a simple encoder architecture using only small sets of names sampled from high-level biomedical concepts. Our encoder post-processes pretrained representations of biomedical names, and is effective for various types of input representations, both domain-specific or unsupervised. We validate our proposed few-shot learning approach on multiple biomedical relatedness benchmarks, and show that it allows for continual learning, where we accumulate information from various conceptual hierarchies to consistently improve encoder performance. Given these findings, we propose our approach as a low-cost alternative for exploring the impact of conceptual distinctions on robust biomedical name representations.
Anthology ID:
2021.bionlp-1.3
Volume:
Proceedings of the 20th Workshop on Biomedical Language Processing
Month:
June
Year:
2021
Address:
Online
Editors:
Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, Junichi Tsujii
Venue:
BioNLP
SIG:
SIGBIOMED
Publisher:
Association for Computational Linguistics
Note:
Pages:
23–29
Language:
URL:
https://aclanthology.org/2021.bionlp-1.3
DOI:
10.18653/v1/2021.bionlp-1.3
Bibkey:
Cite (ACL):
Pieter Fivez, Simon Suster, and Walter Daelemans. 2021. Scalable Few-Shot Learning of Robust Biomedical Name Representations. In Proceedings of the 20th Workshop on Biomedical Language Processing, pages 23–29, Online. Association for Computational Linguistics.
Cite (Informal):
Scalable Few-Shot Learning of Robust Biomedical Name Representations (Fivez et al., BioNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.bionlp-1.3.pdf
Code
 clips/fewshot-biomedical-names
Data
EHR-Rel