Farewell Freebase: Migrating the SimpleQuestions Dataset to DBpedia

Michael Azmy, Peng Shi, Jimmy Lin, Ihab Ilyas


Abstract
Question answering over knowledge graphs is an important problem of interest both commercially and academically. There is substantial interest in the class of natural language questions that can be answered via the lookup of a single fact, driven by the availability of the popular SimpleQuestions dataset. The problem with this dataset, however, is that answer triples are provided from Freebase, which has been defunct for several years. As a result, it is difficult to build “real-world” question answering systems that are operationally deployable. Furthermore, a defunct knowledge graph means that much of the infrastructure for querying, browsing, and manipulating triples no longer exists. To address this problem, we present SimpleDBpediaQA, a new benchmark dataset for simple question answering over knowledge graphs that was created by mapping SimpleQuestions entities and predicates from Freebase to DBpedia. Although this mapping is conceptually straightforward, there are a number of nuances that make the task non-trivial, owing to the different conceptual organizations of the two knowledge graphs. To lay the foundation for future research using this dataset, we leverage recent work to provide simple yet strong baselines with and without neural networks.
Anthology ID:
C18-1178
Volume:
Proceedings of the 27th International Conference on Computational Linguistics
Month:
August
Year:
2018
Address:
Santa Fe, New Mexico, USA
Editors:
Emily M. Bender, Leon Derczynski, Pierre Isabelle
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2093–2103
Language:
URL:
https://aclanthology.org/C18-1178
DOI:
Bibkey:
Cite (ACL):
Michael Azmy, Peng Shi, Jimmy Lin, and Ihab Ilyas. 2018. Farewell Freebase: Migrating the SimpleQuestions Dataset to DBpedia. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2093–2103, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Cite (Informal):
Farewell Freebase: Migrating the SimpleQuestions Dataset to DBpedia (Azmy et al., COLING 2018)
Copy Citation:
PDF:
https://aclanthology.org/C18-1178.pdf
Code
 castorini/SimpleDBpediaQA
Data
SimpleDBpediaQASimpleQuestions