Obtaining referential word meanings from visual and distributional information: Experiments on object naming

Sina Zarrieß, David Schlangen


Abstract
We investigate object naming, which is an important sub-task of referring expression generation on real-world images. As opposed to mutually exclusive labels used in object recognition, object names are more flexible, subject to communicative preferences and semantically related to each other. Therefore, we investigate models of referential word meaning that link visual to lexical information which we assume to be given through distributional word embeddings. We present a model that learns individual predictors for object names that link visual and distributional aspects of word meaning during training. We show that this is particularly beneficial for zero-shot learning, as compared to projecting visual objects directly into the distributional space. In a standard object naming task, we find that different ways of combining lexical and visual information achieve very similar performance, though experiments on model combination suggest that they capture complementary aspects of referential meaning.
Anthology ID:
P17-1023
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Editors:
Regina Barzilay, Min-Yen Kan
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
243–254
Language:
URL:
https://aclanthology.org/P17-1023
DOI:
10.18653/v1/P17-1023
Bibkey:
Cite (ACL):
Sina Zarrieß and David Schlangen. 2017. Obtaining referential word meanings from visual and distributional information: Experiments on object naming. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 243–254, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Obtaining referential word meanings from visual and distributional information: Experiments on object naming (Zarrieß & Schlangen, ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-1023.pdf
Video:
 https://aclanthology.org/P17-1023.mp4
Data
ImageNet