<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zhong-etal-2025-crab">
    <titleInfo>
        <title>CRAB: A Benchmark for Evaluating Curation of Retrieval-Augmented LLMs in Biomedicine</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hanmeng</namePart>
        <namePart type="family">Zhong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Linqing</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Wentao</namePart>
        <namePart type="family">Wu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Weilei</namePart>
        <namePart type="family">Wang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Saloni</namePart>
            <namePart type="family">Potdar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lina</namePart>
            <namePart type="family">Rojas-Barahona</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sebastien</namePart>
            <namePart type="family">Montella</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Suzhou (China)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-333-3</identifier>
    </relatedItem>
    <abstract>Recent development in Retrieval-Augmented Large Language Models (LLMs) have shown great promise in biomedical applications. However, a critical gap persists in reliably evaluating their curation ability—the process by which models select and integrate relevant references while filtering out noise. To address this, we introduce the benchmark for Curation of Retrieval-Augmented LLMs in Biomedicine (CRAB), the first multilingual benchmark tailored for evaluating the biomedical curation of retrieval-augmented LLMs, available in English, French, German and Chinese. By incorporating a novel citation-based evaluation metric, CRAB quantifies the curation performance of retrieval-augmented LLMs in biomedicine. Experimental results reveal significant discrepancies in the curation performance of mainstream LLMs, underscoring the urgent need to improve it in the domain of biomedicine.</abstract>
    <identifier type="citekey">zhong-etal-2025-crab</identifier>
    <location>
        <url>https://aclanthology.org/2025.emnlp-industry.3/</url>
    </location>
    <part>
        <date>2025-11</date>
        <extent unit="page">
            <start>34</start>
            <end>49</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T CRAB: A Benchmark for Evaluating Curation of Retrieval-Augmented LLMs in Biomedicine
%A Zhong, Hanmeng
%A Chen, Linqing
%A Wu, Wentao
%A Wang, Weilei
%Y Potdar, Saloni
%Y Rojas-Barahona, Lina
%Y Montella, Sebastien
%S Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track
%D 2025
%8 November
%I Association for Computational Linguistics
%C Suzhou (China)
%@ 979-8-89176-333-3
%F zhong-etal-2025-crab
%X Recent development in Retrieval-Augmented Large Language Models (LLMs) have shown great promise in biomedical applications. However, a critical gap persists in reliably evaluating their curation ability—the process by which models select and integrate relevant references while filtering out noise. To address this, we introduce the benchmark for Curation of Retrieval-Augmented LLMs in Biomedicine (CRAB), the first multilingual benchmark tailored for evaluating the biomedical curation of retrieval-augmented LLMs, available in English, French, German and Chinese. By incorporating a novel citation-based evaluation metric, CRAB quantifies the curation performance of retrieval-augmented LLMs in biomedicine. Experimental results reveal significant discrepancies in the curation performance of mainstream LLMs, underscoring the urgent need to improve it in the domain of biomedicine.
%U https://aclanthology.org/2025.emnlp-industry.3/
%P 34-49

Markdown (Informal)

[CRAB: A Benchmark for Evaluating Curation of Retrieval-Augmented LLMs in Biomedicine](https://aclanthology.org/2025.emnlp-industry.3/) (Zhong et al., EMNLP 2025)

• CRAB: A Benchmark for Evaluating Curation of Retrieval-Augmented LLMs in Biomedicine (Zhong et al., EMNLP 2025)

ACL

• Hanmeng Zhong, Linqing Chen, Wentao Wu, and Weilei Wang. 2025. CRAB: A Benchmark for Evaluating Curation of Retrieval-Augmented LLMs in Biomedicine. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 34–49, Suzhou (China). Association for Computational Linguistics.