Improving Relation Extraction by Sequence-to-sequence-based Dependency Parsing Pre-training

Masaki Asada, Makoto Miwa


Abstract
Relation extraction is a crucial natural language processing task that extracts relational triplets from raw text. Syntactic dependencies information has shown its effectiveness for relation extraction tasks. However, in most existing studies, dependency information is used only for traditional encoder-only-based relation extraction, not for generative sequence-to-sequence (seq2seq)-based relation extraction. In this study, we propose a syntax-aware seq2seq pre-trained model for seq2seq-based relation extraction. The model incorporates dependency information into a seq2seq pre-trained language model by continual pre-training with a seq2seq-based dependency parsing task. Experimental results on two widely used relation extraction benchmark datasets show that dependency parsing pre-training can improve the relation extraction performance.
Anthology ID:
2025.coling-main.473
Volume:
Proceedings of the 31st International Conference on Computational Linguistics
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7099–7105
Language:
URL:
https://aclanthology.org/2025.coling-main.473/
DOI:
Bibkey:
Cite (ACL):
Masaki Asada and Makoto Miwa. 2025. Improving Relation Extraction by Sequence-to-sequence-based Dependency Parsing Pre-training. In Proceedings of the 31st International Conference on Computational Linguistics, pages 7099–7105, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
Improving Relation Extraction by Sequence-to-sequence-based Dependency Parsing Pre-training (Asada & Miwa, COLING 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.coling-main.473.pdf