Multiˆ2OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT

Youngbin Ro, Yukyung Lee, Pilsung Kang


Abstract
In this paper, we propose Multi2OIE, which performs open information extraction (open IE) by combining BERT with multi-head attention. Our model is a sequence-labeling system with an efficient and effective argument extraction method. We use a query, key, and value setting inspired by the Multimodal Transformer to replace the previously used bidirectional long short-term memory architecture with multi-head attention. Multi2OIE outperforms existing sequence-labeling systems with high computational efficiency on two benchmark evaluation datasets, Re-OIE2016 and CaRB. Additionally, we apply the proposed method to multilingual open IE using multilingual BERT. Experimental results on new benchmark datasets introduced for two languages (Spanish and Portuguese) demonstrate that our model outperforms other multilingual systems without training data for the target languages.
Anthology ID:
2020.findings-emnlp.99
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2020
Month:
November
Year:
2020
Address:
Online
Editors:
Trevor Cohn, Yulan He, Yang Liu
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1107–1117
Language:
URL:
https://aclanthology.org/2020.findings-emnlp.99
DOI:
10.18653/v1/2020.findings-emnlp.99
Bibkey:
Cite (ACL):
Youngbin Ro, Yukyung Lee, and Pilsung Kang. 2020. Multiˆ2OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1107–1117, Online. Association for Computational Linguistics.
Cite (Informal):
Multiˆ2OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT (Ro et al., Findings 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.findings-emnlp.99.pdf
Optional supplementary material:
 2020.findings-emnlp.99.OptionalSupplementaryMaterial.zip
Video:
 https://slideslive.com/38940647
Code
 youngbin-ro/Multi2OIE
Data
CaRBOIE2016