<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="nakarmi-etal-2026-relevance">
    <titleInfo>
        <title>Relevance is \mathitnot enough: A Communication-Oriented Retrieval System for Consequential Scientific Question Answering</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Avina</namePart>
        <namePart type="family">Nakarmi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Naga</namePart>
        <namePart type="given">Datha</namePart>
        <namePart type="given">Saikiran</namePart>
        <namePart type="family">Battula</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anthony</namePart>
        <namePart type="family">Diaz</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Aritra</namePart>
        <namePart type="family">Dasgupta</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Pepa</namePart>
            <namePart type="family">Atanasova</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Gholamreza</namePart>
            <namePart type="family">Haffari</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Minghan</namePart>
            <namePart type="family">Wang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-460-6</identifier>
    </relatedItem>
    <abstract>AI systems increasingly answer scientific questions about health, safety, and the environment. But most retrieval-augmented generation systems are tuned to provide factually correct, on-topic answers rather than to help non-experts understand what those answers mean for their lives and decisions. We focus on consequential scientific questions whose results directly shape people’s lives and study them through a public water-quality communication system, where residents and community leaders interpret the findings and choose actions. Their experiences show that on-topic answers can still be insufficient without explanation and context and that the emotional weight of risk information cannot be ignored. Our system first classifies each question by reasoning type (for example, causal versus policy-based), then generates follow-up questions to identify missing evidence and retrieve it. One component clearly distinguishes between what is known and what is uncertain, while another rewrites scientific details into accessible language, using persona-based styles, such as a caring neighbor or an administrative official, to adapt tone and readability. Ablations on over 160 questions show that the system uses an order of magnitude less context and, in several configurations, improves human-rated completeness. A completeness metric co-designed with community members and a fine-tuned learned judge reveal that standard relevance scores explain about 1% of variation in human completeness ratings, and even the tuned judge only moderately aligns with humans, indicating that completeness is a distinct human-centered objective that current metrics do not reliably capture.</abstract>
    <identifier type="citekey">nakarmi-etal-2026-relevance</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-industry.216/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>3614</start>
            <end>3625</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Relevance is \mathitnot enough: A Communication-Oriented Retrieval System for Consequential Scientific Question Answering
%A Nakarmi, Avina
%A Battula, Naga Datha Saikiran
%A Diaz, Anthony
%A Dasgupta, Aritra
%Y Atanasova, Pepa
%Y Haffari, Gholamreza
%Y Wang, Minghan
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-460-6
%F nakarmi-etal-2026-relevance
%X AI systems increasingly answer scientific questions about health, safety, and the environment. But most retrieval-augmented generation systems are tuned to provide factually correct, on-topic answers rather than to help non-experts understand what those answers mean for their lives and decisions. We focus on consequential scientific questions whose results directly shape people’s lives and study them through a public water-quality communication system, where residents and community leaders interpret the findings and choose actions. Their experiences show that on-topic answers can still be insufficient without explanation and context and that the emotional weight of risk information cannot be ignored. Our system first classifies each question by reasoning type (for example, causal versus policy-based), then generates follow-up questions to identify missing evidence and retrieve it. One component clearly distinguishes between what is known and what is uncertain, while another rewrites scientific details into accessible language, using persona-based styles, such as a caring neighbor or an administrative official, to adapt tone and readability. Ablations on over 160 questions show that the system uses an order of magnitude less context and, in several configurations, improves human-rated completeness. A completeness metric co-designed with community members and a fine-tuned learned judge reveal that standard relevance scores explain about 1% of variation in human completeness ratings, and even the tuned judge only moderately aligns with humans, indicating that completeness is a distinct human-centered objective that current metrics do not reliably capture.
%U https://aclanthology.org/2026.emnlp-industry.216/
%P 3614-3625

Markdown (Informal)

[Relevance is not enough: A Communication-Oriented Retrieval System for Consequential Scientific Question Answering](https://aclanthology.org/2026.emnlp-industry.216/) (Nakarmi et al., EMNLP 2026)

• Relevance is not enough: A Communication-Oriented Retrieval System for Consequential Scientific Question Answering (Nakarmi et al., EMNLP 2026)

ACL

• Avina Nakarmi, Naga Datha Saikiran Battula, Anthony Diaz, and Aritra Dasgupta. 2026. Relevance is not enough: A Communication-Oriented Retrieval System for Consequential Scientific Question Answering. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 3614–3625, Budapest, Hungary. Association for Computational Linguistics.