<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yi-etal-2020-improving">
    <titleInfo>
        <title>Improving Image Captioning Evaluation by Considering Inter References Variance</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yanzhi</namePart>
        <namePart type="family">Yi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hangyu</namePart>
        <namePart type="family">Deng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jinglu</namePart>
        <namePart type="family">Hu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Dan</namePart>
            <namePart type="family">Jurafsky</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Joyce</namePart>
            <namePart type="family">Chai</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Natalie</namePart>
            <namePart type="family">Schluter</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Joel</namePart>
            <namePart type="family">Tetreault</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Evaluating image captions is very challenging partially due to the fact that there are multiple correct captions for every single image. Most of the existing one-to-one metrics operate by penalizing mismatches between reference and generative caption without considering the intrinsic variance between ground truth captions. It usually leads to over-penalization and thus a bad correlation to human judgment. Recently, the latest one-to-one metric BERTScore can achieve high human correlation in system-level tasks while some issues can be fixed for better performance. In this paper, we propose a novel metric based on BERTScore that could handle such a challenge and extend BERTScore with a few new features appropriately for image captioning evaluation. The experimental results show that our metric achieves state-of-the-art human judgment correlation.</abstract>
    <identifier type="citekey">yi-etal-2020-improving</identifier>
    <identifier type="doi">10.18653/v1/2020.acl-main.93</identifier>
    <location>
        <url>https://aclanthology.org/2020.acl-main.93</url>
    </location>
    <part>
        <date>2020-07</date>
        <extent unit="page">
            <start>985</start>
            <end>994</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Improving Image Captioning Evaluation by Considering Inter References Variance
%A Yi, Yanzhi
%A Deng, Hangyu
%A Hu, Jinglu
%Y Jurafsky, Dan
%Y Chai, Joyce
%Y Schluter, Natalie
%Y Tetreault, Joel
%S Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
%D 2020
%8 July
%I Association for Computational Linguistics
%C Online
%F yi-etal-2020-improving
%X Evaluating image captions is very challenging partially due to the fact that there are multiple correct captions for every single image. Most of the existing one-to-one metrics operate by penalizing mismatches between reference and generative caption without considering the intrinsic variance between ground truth captions. It usually leads to over-penalization and thus a bad correlation to human judgment. Recently, the latest one-to-one metric BERTScore can achieve high human correlation in system-level tasks while some issues can be fixed for better performance. In this paper, we propose a novel metric based on BERTScore that could handle such a challenge and extend BERTScore with a few new features appropriately for image captioning evaluation. The experimental results show that our metric achieves state-of-the-art human judgment correlation.
%R 10.18653/v1/2020.acl-main.93
%U https://aclanthology.org/2020.acl-main.93
%U https://doi.org/10.18653/v1/2020.acl-main.93
%P 985-994

Markdown (Informal)

[Improving Image Captioning Evaluation by Considering Inter References Variance](https://aclanthology.org/2020.acl-main.93) (Yi et al., ACL 2020)

• Improving Image Captioning Evaluation by Considering Inter References Variance (Yi et al., ACL 2020)

ACL

• Yanzhi Yi, Hangyu Deng, and Jinglu Hu. 2020. Improving Image Captioning Evaluation by Considering Inter References Variance. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 985–994, Online. Association for Computational Linguistics.