GAIA: A Fine-grained Multimedia Knowledge Extraction System

Manling Li, Alireza Zareian, Ying Lin, Xiaoman Pan, Spencer Whitehead, Brian Chen, Bo Wu, Heng Ji, Shih-Fu Chang, Clare Voss, Daniel Napierski, Marjorie Freedman


Abstract
We present the first comprehensive, open source multimedia knowledge extraction system that takes a massive stream of unstructured, heterogeneous multimedia data from various sources and languages as input, and creates a coherent, structured knowledge base, indexing entities, relations, and events, following a rich, fine-grained ontology. Our system, GAIA, enables seamless search of complex graph queries, and retrieves multimedia evidence including text, images and videos. GAIA achieves top performance at the recent NIST TAC SM-KBP2019 evaluation. The system is publicly available at GitHub and DockerHub, with a narrated video that documents the system.
Anthology ID:
2020.acl-demos.11
Original:
2020.acl-demos.11v1
Version 2:
2020.acl-demos.11v2
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations
Month:
July
Year:
2020
Address:
Online
Editors:
Asli Celikyilmaz, Tsung-Hsien Wen
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
77–86
Language:
URL:
https://aclanthology.org/2020.acl-demos.11
DOI:
10.18653/v1/2020.acl-demos.11
Award:
 Best Demonstration Paper
Bibkey:
Cite (ACL):
Manling Li, Alireza Zareian, Ying Lin, Xiaoman Pan, Spencer Whitehead, Brian Chen, Bo Wu, Heng Ji, Shih-Fu Chang, Clare Voss, Daniel Napierski, and Marjorie Freedman. 2020. GAIA: A Fine-grained Multimedia Knowledge Extraction System. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 77–86, Online. Association for Computational Linguistics.
Cite (Informal):
GAIA: A Fine-grained Multimedia Knowledge Extraction System (Li et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-demos.11.pdf
Video:
 http://slideslive.com/38928613