<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kawada-etal-2026-recall">
    <titleInfo>
        <title>Recall Is Not Completeness: Retrieving All the Evidence a Question Needs</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sebastien</namePart>
        <namePart type="family">Kawada</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Adhiban</namePart>
        <namePart type="family">Arulselvan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Manolis</namePart>
        <namePart type="family">Kellis</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Natural Legal Language Processing Workshop 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Nikolaos</namePart>
            <namePart type="family">Aletras</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ilias</namePart>
            <namePart type="family">Chalkidis</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Leslie</namePart>
            <namePart type="family">Barrett</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Cătălina</namePart>
            <namePart type="family">Goanță</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Daniel</namePart>
            <namePart type="family">Preoțiuc-Pietro</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Gerasimos</namePart>
            <namePart type="family">Spanakis</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Shanshan</namePart>
            <namePart type="family">Xu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-494-1</identifier>
    </relatedItem>
    <abstract>Questions in law, policy, the life sciences, and the humanities often require evidence from many sources, all of which retrieval must find. We prove that recall can be high even when few questions have all of their gold items retrieved, that is, are complete. We demonstrate that completeness strongly affects answer correctness. On HotpotQA questions where Quorum, our learned fusion of diverse retrievers, is the only retriever to retrieve all of the supporting paragraphs, our reader’s exact match rises from 0.212 to 0.553. We find that recall hides poor completeness. On Housing Statute QA questions that cite four or more statutes, Quorum’s top-ten results include an average of 39.3% of the gold items, yet are complete for only 2.3% of the questions. Recall exceeds completeness by 17- to 81-fold on this subset for all seven systems we evaluate. The candidate pools contain all of the gold items for 65% of these questions, meaning that the reranker causes more failures of completeness than the candidate retrieval. Quorum significantly outperforms the baseline, improving completeness@5 over dense retrieval from 0.453 to 0.670 on MuSiQue and accuracy on the agent-memory benchmark LongMemEval-V2 from 0.385 to 0.530. Benchmarks where multiple items must be retrieved should report completeness conditioned on the number of gold items.</abstract>
    <identifier type="citekey">kawada-etal-2026-recall</identifier>
    <location>
        <url>https://aclanthology.org/2026.nllp-1.22/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>258</start>
            <end>258</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Recall Is Not Completeness: Retrieving All the Evidence a Question Needs
%A Kawada, Sebastien
%A Arulselvan, Adhiban
%A Kellis, Manolis
%Y Aletras, Nikolaos
%Y Chalkidis, Ilias
%Y Barrett, Leslie
%Y Goanță, Cătălina
%Y Preoțiuc-Pietro, Daniel
%Y Spanakis, Gerasimos
%Y Xu, Shanshan
%S Proceedings of the Natural Legal Language Processing Workshop 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-494-1
%F kawada-etal-2026-recall
%X Questions in law, policy, the life sciences, and the humanities often require evidence from many sources, all of which retrieval must find. We prove that recall can be high even when few questions have all of their gold items retrieved, that is, are complete. We demonstrate that completeness strongly affects answer correctness. On HotpotQA questions where Quorum, our learned fusion of diverse retrievers, is the only retriever to retrieve all of the supporting paragraphs, our reader’s exact match rises from 0.212 to 0.553. We find that recall hides poor completeness. On Housing Statute QA questions that cite four or more statutes, Quorum’s top-ten results include an average of 39.3% of the gold items, yet are complete for only 2.3% of the questions. Recall exceeds completeness by 17- to 81-fold on this subset for all seven systems we evaluate. The candidate pools contain all of the gold items for 65% of these questions, meaning that the reranker causes more failures of completeness than the candidate retrieval. Quorum significantly outperforms the baseline, improving completeness@5 over dense retrieval from 0.453 to 0.670 on MuSiQue and accuracy on the agent-memory benchmark LongMemEval-V2 from 0.385 to 0.530. Benchmarks where multiple items must be retrieved should report completeness conditioned on the number of gold items.
%U https://aclanthology.org/2026.nllp-1.22/
%P 258-258

Markdown (Informal)

[Recall Is Not Completeness: Retrieving All the Evidence a Question Needs](https://aclanthology.org/2026.nllp-1.22/) (Kawada et al., NLLP 2026)

• Recall Is Not Completeness: Retrieving All the Evidence a Question Needs (Kawada et al., NLLP 2026)

ACL

• Sebastien Kawada, Adhiban Arulselvan, and Manolis Kellis. 2026. Recall Is Not Completeness: Retrieving All the Evidence a Question Needs. In Proceedings of the Natural Legal Language Processing Workshop 2026, pages 258–258, Budapest, Hungary. Association for Computational Linguistics.