<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="rashid-etal-2024-ecorank">
    <titleInfo>
        <title>EcoRank: Budget-Constrained Text Re-ranking Using Large Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Muhammad</namePart>
        <namePart type="family">Rashid</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jannat</namePart>
        <namePart type="family">Meem</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yue</namePart>
        <namePart type="family">Dong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vagelis</namePart>
        <namePart type="family">Hristidis</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics ACL 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Lun-Wei</namePart>
            <namePart type="family">Ku</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andre</namePart>
            <namePart type="family">Martins</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand and virtual meeting</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Large Language Models (LLMs) have achieved state-of-the-art performance in text re-ranking. This process includes queries and candidate passages in the prompts, utilizing pointwise, listwise, and pairwise prompting strategies. A limitation of these ranking strategies with LLMs is their cost: the process can become expensive due to API charges, which are based on the number of input and output tokens. We study how to maximize the re-ranking performance given a budget, by navigating the vast search spaces of prompt choices, LLM APIs, and budget splits. We propose a suite of budget-constrained methods to perform text re-ranking using a set of LLM APIs. Our most efficient method, called EcoRank, is a two-layered pipeline that jointly optimizes decisions regarding budget allocation across prompt strategies and LLM APIs. Our experimental results on four popular QA and passage reranking datasets show that EcoRank outperforms other budget-aware supervised and unsupervised baselines.</abstract>
    <identifier type="citekey">rashid-etal-2024-ecorank</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-acl.773</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>13049</start>
            <end>13063</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T EcoRank: Budget-Constrained Text Re-ranking Using Large Language Models
%A Rashid, Muhammad
%A Meem, Jannat
%A Dong, Yue
%A Hristidis, Vagelis
%Y Ku, Lun-Wei
%Y Martins, Andre
%Y Srikumar, Vivek
%S Findings of the Association for Computational Linguistics ACL 2024
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand and virtual meeting
%F rashid-etal-2024-ecorank
%X Large Language Models (LLMs) have achieved state-of-the-art performance in text re-ranking. This process includes queries and candidate passages in the prompts, utilizing pointwise, listwise, and pairwise prompting strategies. A limitation of these ranking strategies with LLMs is their cost: the process can become expensive due to API charges, which are based on the number of input and output tokens. We study how to maximize the re-ranking performance given a budget, by navigating the vast search spaces of prompt choices, LLM APIs, and budget splits. We propose a suite of budget-constrained methods to perform text re-ranking using a set of LLM APIs. Our most efficient method, called EcoRank, is a two-layered pipeline that jointly optimizes decisions regarding budget allocation across prompt strategies and LLM APIs. Our experimental results on four popular QA and passage reranking datasets show that EcoRank outperforms other budget-aware supervised and unsupervised baselines.
%U https://aclanthology.org/2024.findings-acl.773
%P 13049-13063

Markdown (Informal)

[EcoRank: Budget-Constrained Text Re-ranking Using Large Language Models](https://aclanthology.org/2024.findings-acl.773) (Rashid et al., Findings 2024)

• EcoRank: Budget-Constrained Text Re-ranking Using Large Language Models (Rashid et al., Findings 2024)

ACL

• Muhammad Rashid, Jannat Meem, Yue Dong, and Vagelis Hristidis. 2024. EcoRank: Budget-Constrained Text Re-ranking Using Large Language Models. In Findings of the Association for Computational Linguistics ACL 2024, pages 13049–13063, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.