<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="uthus-ni-2023-rise">
    <titleInfo>
        <title>RISE: Leveraging Retrieval Techniques for Summarization Evaluation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">David</namePart>
        <namePart type="family">Uthus</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jianmo</namePart>
        <namePart type="family">Ni</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Evaluating automatically-generated text summaries is a challenging task. While there have been many interesting approaches, they still fall short of human evaluations. We present RISE, a new approach for evaluating summaries by leveraging techniques from information retrieval. RISE is first trained as a retrieval task using a dual-encoder retrieval setup, and can then be subsequently utilized for evaluating a generated summary given an input document, without gold reference summaries. RISE is especially well suited when working on new datasets where one may not have reference summaries available for evaluation. We conduct comprehensive experiments on the SummEval benchmark (Fabbri et al., 2021) and a long document summarization benchmark. The results show that RISE consistently achieves higher correlation with human evaluations compared to many past approaches to summarization evaluation. Furthermore, RISE also demonstrates data-efficiency and generalizability across languages.</abstract>
    <identifier type="citekey">uthus-ni-2023-rise</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-acl.865</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-acl.865</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>13697</start>
            <end>13709</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T RISE: Leveraging Retrieval Techniques for Summarization Evaluation
%A Uthus, David
%A Ni, Jianmo
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Findings of the Association for Computational Linguistics: ACL 2023
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F uthus-ni-2023-rise
%X Evaluating automatically-generated text summaries is a challenging task. While there have been many interesting approaches, they still fall short of human evaluations. We present RISE, a new approach for evaluating summaries by leveraging techniques from information retrieval. RISE is first trained as a retrieval task using a dual-encoder retrieval setup, and can then be subsequently utilized for evaluating a generated summary given an input document, without gold reference summaries. RISE is especially well suited when working on new datasets where one may not have reference summaries available for evaluation. We conduct comprehensive experiments on the SummEval benchmark (Fabbri et al., 2021) and a long document summarization benchmark. The results show that RISE consistently achieves higher correlation with human evaluations compared to many past approaches to summarization evaluation. Furthermore, RISE also demonstrates data-efficiency and generalizability across languages.
%R 10.18653/v1/2023.findings-acl.865
%U https://aclanthology.org/2023.findings-acl.865
%U https://doi.org/10.18653/v1/2023.findings-acl.865
%P 13697-13709

Markdown (Informal)

[RISE: Leveraging Retrieval Techniques for Summarization Evaluation](https://aclanthology.org/2023.findings-acl.865) (Uthus & Ni, Findings 2023)

• RISE: Leveraging Retrieval Techniques for Summarization Evaluation (Uthus & Ni, Findings 2023)

ACL

• David Uthus and Jianmo Ni. 2023. RISE: Leveraging Retrieval Techniques for Summarization Evaluation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13697–13709, Toronto, Canada. Association for Computational Linguistics.