<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="guo-vosoughi-2023-length">
    <titleInfo>
        <title>Length Does Matter: Summary Length can Bias Summarization Metrics</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Xiaobo</namePart>
        <namePart type="family">Guo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Soroush</namePart>
        <namePart type="family">Vosoughi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Establishing the characteristics of an effective summary is a complicated and often subjective endeavor. Consequently, the development of metrics for the summarization task has become a dynamic area of research within natural language processing. In this paper, we reveal that existing summarization metrics exhibit a bias toward the length of generated summaries. Our thorough experiments, conducted on a variety of datasets, metrics, and models, substantiate these findings. The results indicate that most metrics tend to favor longer summaries, even after accounting for other factors. To address this issue, we introduce a Bayesian normalization technique that effectively diminishes this bias. We demonstrate that our approach significantly improves the concordance between human annotators and the majority of metrics in terms of summary coherence.</abstract>
    <identifier type="citekey">guo-vosoughi-2023-length</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.984</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.984</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>15869</start>
            <end>15879</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Length Does Matter: Summary Length can Bias Summarization Metrics
%A Guo, Xiaobo
%A Vosoughi, Soroush
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F guo-vosoughi-2023-length
%X Establishing the characteristics of an effective summary is a complicated and often subjective endeavor. Consequently, the development of metrics for the summarization task has become a dynamic area of research within natural language processing. In this paper, we reveal that existing summarization metrics exhibit a bias toward the length of generated summaries. Our thorough experiments, conducted on a variety of datasets, metrics, and models, substantiate these findings. The results indicate that most metrics tend to favor longer summaries, even after accounting for other factors. To address this issue, we introduce a Bayesian normalization technique that effectively diminishes this bias. We demonstrate that our approach significantly improves the concordance between human annotators and the majority of metrics in terms of summary coherence.
%R 10.18653/v1/2023.emnlp-main.984
%U https://aclanthology.org/2023.emnlp-main.984
%U https://doi.org/10.18653/v1/2023.emnlp-main.984
%P 15869-15879

Markdown (Informal)

[Length Does Matter: Summary Length can Bias Summarization Metrics](https://aclanthology.org/2023.emnlp-main.984) (Guo & Vosoughi, EMNLP 2023)

• Length Does Matter: Summary Length can Bias Summarization Metrics (Guo & Vosoughi, EMNLP 2023)

ACL

• Xiaobo Guo and Soroush Vosoughi. 2023. Length Does Matter: Summary Length can Bias Summarization Metrics. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15869–15879, Singapore. Association for Computational Linguistics.