<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="cano-bojar-2019-efficiency">
    <titleInfo>
        <title>Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Erion</namePart>
        <namePart type="family">Çano</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ondřej</namePart>
        <namePart type="family">Bojar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-oct–nov</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 12th International Conference on Natural Language Generation</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Kees</namePart>
            <namePart type="family">van Deemter</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Chenghua</namePart>
            <namePart type="family">Lin</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hiroya</namePart>
            <namePart type="family">Takamura</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Tokyo, Japan</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Using data-driven models for solving text summarization or similar tasks has become very common in the last years. Yet most of the studies report basic accuracy scores only, and nothing is known about the ability of the proposed models to improve when trained on more data. In this paper, we define and propose three data efficiency metrics: data score efficiency, data time deficiency and overall data efficiency. We also propose a simple scheme that uses those metrics and apply it for a more comprehensive evaluation of popular methods on text summarization and title generation tasks. For the latter task, we process and release a huge collection of 35 million abstract-title pairs from scientific articles. Our results reveal that among the tested models, the Transformer is the most efficient on both tasks.</abstract>
    <identifier type="citekey">cano-bojar-2019-efficiency</identifier>
    <identifier type="doi">10.18653/v1/W19-8630</identifier>
    <location>
        <url>https://aclanthology.org/W19-8630</url>
    </location>
    <part>
        <date>2019-oct–nov</date>
        <extent unit="page">
            <start>229</start>
            <end>239</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study
%A Çano, Erion
%A Bojar, Ondřej
%Y van Deemter, Kees
%Y Lin, Chenghua
%Y Takamura, Hiroya
%S Proceedings of the 12th International Conference on Natural Language Generation
%D 2019
%8 oct–nov
%I Association for Computational Linguistics
%C Tokyo, Japan
%F cano-bojar-2019-efficiency
%X Using data-driven models for solving text summarization or similar tasks has become very common in the last years. Yet most of the studies report basic accuracy scores only, and nothing is known about the ability of the proposed models to improve when trained on more data. In this paper, we define and propose three data efficiency metrics: data score efficiency, data time deficiency and overall data efficiency. We also propose a simple scheme that uses those metrics and apply it for a more comprehensive evaluation of popular methods on text summarization and title generation tasks. For the latter task, we process and release a huge collection of 35 million abstract-title pairs from scientific articles. Our results reveal that among the tested models, the Transformer is the most efficient on both tasks.
%R 10.18653/v1/W19-8630
%U https://aclanthology.org/W19-8630
%U https://doi.org/10.18653/v1/W19-8630
%P 229-239

Markdown (Informal)

[Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study](https://aclanthology.org/W19-8630) (Çano & Bojar, INLG 2019)

• Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study (Çano & Bojar, INLG 2019)

ACL

• Erion Çano and Ondřej Bojar. 2019. Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study. In Proceedings of the 12th International Conference on Natural Language Generation, pages 229–239, Tokyo, Japan. Association for Computational Linguistics.