Relation Extraction Using Multiple Pre-Training Models in Biomedical Domain

Satoshi Hiai, Kazutaka Shimada, Taiki Watanabe, Akiva Miura, Tomoya Iwakura


Abstract
The number of biomedical documents is increasing rapidly. Accordingly, a demand for extracting knowledge from large-scale biomedical texts is also increasing. BERT-based models are known for their high performance in various tasks. However, it is often computationally expensive. A high-end GPU environment is not available in many situations. To attain both high accuracy and fast extraction speed, we propose combinations of simpler pre-trained models. Our method outperforms the latest state-of-the-art model and BERT-based models on the GAD corpus. In addition, our method shows approximately three times faster extraction speed than the BERT-based models on the ChemProt corpus and reduces the memory size to one sixth of the BERT ones.
Anthology ID:
2021.ranlp-1.60
Volume:
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
Month:
September
Year:
2021
Address:
Held Online
Editors:
Ruslan Mitkov, Galia Angelova
Venue:
RANLP
SIG:
Publisher:
INCOMA Ltd.
Note:
Pages:
530–537
Language:
URL:
https://aclanthology.org/2021.ranlp-1.60
DOI:
Bibkey:
Cite (ACL):
Satoshi Hiai, Kazutaka Shimada, Taiki Watanabe, Akiva Miura, and Tomoya Iwakura. 2021. Relation Extraction Using Multiple Pre-Training Models in Biomedical Domain. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 530–537, Held Online. INCOMA Ltd..
Cite (Informal):
Relation Extraction Using Multiple Pre-Training Models in Biomedical Domain (Hiai et al., RANLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.ranlp-1.60.pdf