A Corpus of Natural Language for Visual Reasoning

Alane Suhr, Mike Lewis, James Yeh, Yoav Artzi


Abstract
We present a new visual reasoning language dataset, containing 92,244 pairs of examples of natural statements grounded in synthetic images with 3,962 unique sentences. We describe a method of crowdsourcing linguistically-diverse data, and present an analysis of our data. The data demonstrates a broad set of linguistic phenomena, requiring visual and set-theoretic reasoning. We experiment with various models, and show the data presents a strong challenge for future research.
Anthology ID:
P17-2034
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Editors:
Regina Barzilay, Min-Yen Kan
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
217–223
Language:
URL:
https://aclanthology.org/P17-2034
DOI:
10.18653/v1/P17-2034
Bibkey:
Cite (ACL):
Alane Suhr, Mike Lewis, James Yeh, and Yoav Artzi. 2017. A Corpus of Natural Language for Visual Reasoning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 217–223, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
A Corpus of Natural Language for Visual Reasoning (Suhr et al., ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-2034.pdf
Note:
 P17-2034.Notes.pdf
Video:
 https://aclanthology.org/P17-2034.mp4
Data
NLVRCLEVRSHAPES