<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="vasilyev-etal-2020-fill">
    <titleInfo>
        <title>Fill in the BLANC: Human-free quality estimation of document summaries</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Oleg</namePart>
        <namePart type="family">Vasilyev</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vedant</namePart>
        <namePart type="family">Dharnidharka</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">John</namePart>
        <namePart type="family">Bohannon</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Steffen</namePart>
            <namePart type="family">Eger</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yang</namePart>
            <namePart type="family">Gao</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Maxime</namePart>
            <namePart type="family">Peyrard</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Wei</namePart>
            <namePart type="family">Zhao</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Eduard</namePart>
            <namePart type="family">Hovy</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We present BLANC, a new approach to the automatic estimation of document summary quality. Our goal is to measure the functional performance of a summary with an objective, reproducible, and fully automated method. Our approach achieves this by measuring the performance boost gained by a pre-trained language model with access to a document summary while carrying out its language understanding task on the document’s text. We present evidence that BLANC scores have as good correlation with human evaluations as do the ROUGE family of summary quality measurements. And unlike ROUGE, the BLANC method does not require human-written reference summaries, allowing for fully human-free summary quality estimation.</abstract>
    <identifier type="citekey">vasilyev-etal-2020-fill</identifier>
    <identifier type="doi">10.18653/v1/2020.eval4nlp-1.2</identifier>
    <location>
        <url>https://aclanthology.org/2020.eval4nlp-1.2</url>
    </location>
    <part>
        <date>2020-11</date>
        <extent unit="page">
            <start>11</start>
            <end>20</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Fill in the BLANC: Human-free quality estimation of document summaries
%A Vasilyev, Oleg
%A Dharnidharka, Vedant
%A Bohannon, John
%Y Eger, Steffen
%Y Gao, Yang
%Y Peyrard, Maxime
%Y Zhao, Wei
%Y Hovy, Eduard
%S Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems
%D 2020
%8 November
%I Association for Computational Linguistics
%C Online
%F vasilyev-etal-2020-fill
%X We present BLANC, a new approach to the automatic estimation of document summary quality. Our goal is to measure the functional performance of a summary with an objective, reproducible, and fully automated method. Our approach achieves this by measuring the performance boost gained by a pre-trained language model with access to a document summary while carrying out its language understanding task on the document’s text. We present evidence that BLANC scores have as good correlation with human evaluations as do the ROUGE family of summary quality measurements. And unlike ROUGE, the BLANC method does not require human-written reference summaries, allowing for fully human-free summary quality estimation.
%R 10.18653/v1/2020.eval4nlp-1.2
%U https://aclanthology.org/2020.eval4nlp-1.2
%U https://doi.org/10.18653/v1/2020.eval4nlp-1.2
%P 11-20

Markdown (Informal)

[Fill in the BLANC: Human-free quality estimation of document summaries](https://aclanthology.org/2020.eval4nlp-1.2) (Vasilyev et al., Eval4NLP 2020)

• Fill in the BLANC: Human-free quality estimation of document summaries (Vasilyev et al., Eval4NLP 2020)

ACL

• Oleg Vasilyev, Vedant Dharnidharka, and John Bohannon. 2020. Fill in the BLANC: Human-free quality estimation of document summaries. In Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems, pages 11–20, Online. Association for Computational Linguistics.