On Aligning OpenIE Extractions with Knowledge Bases: A Case Study

Kiril Gashteovski, Rainer Gemulla, Bhushan Kotnis, Sven Hertling, Christian Meilicke


Abstract
Open information extraction (OIE) is the task of extracting relations and their corresponding arguments from a natural language text in un- supervised manner. Outputs of such systems are used for downstream tasks such as ques- tion answering and automatic knowledge base (KB) construction. Many of these downstream tasks rely on aligning OIE triples with refer- ence KBs. Such alignments are usually eval- uated w.r.t. a specific downstream task and, to date, no direct manual evaluation of such alignments has been performed. In this paper, we directly evaluate how OIE triples from the OPIEC corpus are related to the DBpedia KB w.r.t. information content. First, we investigate OPIEC triples and DBpedia facts having the same arguments by comparing the information on the OIE surface relation with the KB rela- tion. Second, we evaluate the expressibility of general OPIEC triples in DBpedia. We in- vestigate whether—and, if so, how—a given OIE triple can be mapped to a single KB fact. We found that such mappings are not always possible because the information in the OIE triples tends to be more specific. Our evalua- tion suggests, however, that significant part of OIE triples can be expressed by means of KB formulas instead of individual facts.
Anthology ID:
2020.eval4nlp-1.14
Volume:
Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems
Month:
November
Year:
2020
Address:
Online
Editors:
Steffen Eger, Yang Gao, Maxime Peyrard, Wei Zhao, Eduard Hovy
Venue:
Eval4NLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
143–154
Language:
URL:
https://aclanthology.org/2020.eval4nlp-1.14
DOI:
10.18653/v1/2020.eval4nlp-1.14
Bibkey:
Cite (ACL):
Kiril Gashteovski, Rainer Gemulla, Bhushan Kotnis, Sven Hertling, and Christian Meilicke. 2020. On Aligning OpenIE Extractions with Knowledge Bases: A Case Study. In Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems, pages 143–154, Online. Association for Computational Linguistics.
Cite (Informal):
On Aligning OpenIE Extractions with Knowledge Bases: A Case Study (Gashteovski et al., Eval4NLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.eval4nlp-1.14.pdf
Video:
 https://slideslive.com/38939720
Data
DBpediaOPIEC