How Much do Knowledge Graphs Impact Transformer Models for Extracting Biomedical Events?

Laura Zanella, Yannick Toussaint


Abstract
Biomedical event extraction can be divided into three main subtasks; (1) biomedical event trigger detection, (2) biomedical argument identification and (3) event construction. This work focuses in the two first subtasks. For the first subtask we analyze a set of transformer language models that are commonly used in the biomedical domain to evaluate and compare their capacity for event trigger detection. We fine-tune the models using seven manually annotated corpora to assess their performance in different biomedical subdomains. SciBERT emerged as the highest performing model, presenting a slight improvement compared to baseline models. Then, for the second subtask we construct a knowledge graph (KG) from the biomedical corpora and integrate its KG embeddings to SciBERT to enrich its semantic information. We demonstrate that adding the KG embeddings to the model improves the argument identification performance by around 20 %, and by around 15 % compared to two baseline models. Our results suggest that fine-tuning a transformer model that is pretrained from scratch with biomedical and general data allows to detect event triggers and identify arguments covering different biomedical subdomains, and therefore improving its generalization. Furthermore, the integration of KG embeddings into the model can significantly improve the performance of biomedical event argument identification, outperforming the results of baseline models.
Anthology ID:
2023.bionlp-1.12
Volume:
The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Dina Demner-fushman, Sophia Ananiadou, Kevin Cohen
Venue:
BioNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
145–155
Language:
URL:
https://aclanthology.org/2023.bionlp-1.12
DOI:
10.18653/v1/2023.bionlp-1.12
Bibkey:
Cite (ACL):
Laura Zanella and Yannick Toussaint. 2023. How Much do Knowledge Graphs Impact Transformer Models for Extracting Biomedical Events?. In The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, pages 145–155, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
How Much do Knowledge Graphs Impact Transformer Models for Extracting Biomedical Events? (Zanella & Toussaint, BioNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.bionlp-1.12.pdf