Fatality Killed the Cat or: BabelPic, a Multimodal Dataset for Non-Concrete Concepts

Agostina Calabrese, Michele Bevilacqua, Roberto Navigli


Abstract
Thanks to the wealth of high-quality annotated images available in popular repositories such as ImageNet, multimodal language-vision research is in full bloom. However, events, feelings and many other kinds of concepts which can be visually grounded are not well represented in current datasets. Nevertheless, we would expect a wide-coverage language understanding system to be able to classify images depicting recess and remorse, not just cats, dogs and bridges. We fill this gap by presenting BabelPic, a hand-labeled dataset built by cleaning the image-synset association found within the BabelNet Lexical Knowledge Base (LKB). BabelPic explicitly targets non-concrete concepts, thus providing refreshing new data for the community. We also show that pre-trained language-vision systems can be used to further expand the resource by exploiting natural language knowledge available in the LKB. BabelPic is available for download at http://babelpic.org.
Anthology ID:
2020.acl-main.425
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4680–4686
Language:
URL:
https://aclanthology.org/2020.acl-main.425
DOI:
10.18653/v1/2020.acl-main.425
Bibkey:
Cite (ACL):
Agostina Calabrese, Michele Bevilacqua, and Roberto Navigli. 2020. Fatality Killed the Cat or: BabelPic, a Multimodal Dataset for Non-Concrete Concepts. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4680–4686, Online. Association for Computational Linguistics.
Cite (Informal):
Fatality Killed the Cat or: BabelPic, a Multimodal Dataset for Non-Concrete Concepts (Calabrese et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.425.pdf
Video:
 http://slideslive.com/38929120