Sofia Nikiforova


pdf bib
Geo-Aware Image Caption Generation
Sofia Nikiforova | Tejaswini Deoskar | Denis Paperno | Yoad Winter
Proceedings of the 28th International Conference on Computational Linguistics

Standard image caption generation systems produce generic descriptions of images and do not utilize any contextual information or world knowledge. In particular, they are unable to generate captions that contain references to the geographic context of an image, for example, the location where a photograph is taken or relevant geographic objects around an image location. In this paper, we develop a geo-aware image caption generation system, which incorporates geographic contextual information into a standard image captioning pipeline. We propose a way to build an image-specific representation of the geographic context and adapt the caption generation network to produce appropriate geographic names in the image descriptions. We evaluate our system on a novel captioning dataset that contains contextualized captions and geographic metadata and achieve substantial improvements in BLEU, ROUGE, METEOR and CIDEr scores. We also introduce a new metric to assess generated geographic references directly and empirically demonstrate our system’s ability to produce captions with relevant and factually accurate geographic referencing.


pdf bib
Rhetorical relations markers in Russian RST Treebank
Svetlana Toldova | Dina Pisarevskaya | Margarita Ananyeva | Maria Kobozeva | Alexander Nasedkin | Sofia Nikiforova | Irina Pavlova | Alexey Shelepov
Proceedings of the 6th Workshop on Recent Advances in RST and Related Formalisms