<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kelber-etal-2026-need">
    <titleInfo>
        <title>Do We Need Bigger Models for Science? Task-Aware Retrieval with Small Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Florian</namePart>
        <namePart type="family">Kelber</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Matthias</namePart>
        <namePart type="family">Jobst</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yuni</namePart>
        <namePart type="family">Susanti</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Michael</namePart>
        <namePart type="family">Färber</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Georg</namePart>
            <namePart type="family">Rehm</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Stefan</namePart>
            <namePart type="family">Dietze</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Danilo</namePart>
            <namePart type="family">Dessi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Diana</namePart>
            <namePart type="family">Maynard</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sonja</namePart>
            <namePart type="family">Schimmler</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resources Association (ELRA)</publisher>
            <place>
                <placeTerm type="text">Palma, Mallorca (Spain)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Scientific knowledge discovery increasingly relies on large language models, yet many existing scholarly assistants depend on proprietary systems with tens or hundreds of billions of parameters. Such reliance limits reproducibility and accessibility for the research community. In this work, we ask a simple question: do we need bigger models for scientific applications? Specifically, we investigate to what extent carefully designed retrieval pipelines can compensate for reduced model scale in scientific applications. We design a lightweight retrieval-augmented framework that performs task-aware routing to select specialized retrieval strategies based on the input query. The system further integrates evidence from full-text scientific papers and structured scholarly metadata, and employs compact instruction-tuned language models to generate responses with citations. We evaluate the framework across several scholarly tasks, focusing on scholarly question answering (QA), including single- and multi-document scenarios, as well as biomedical QA under domain shift and scientific text compression. Our findings demonstrate that retrieval and model scale are complementary rather than interchangeable. While retrieval design can partially compensate for smaller models, model capacity remains important for complex reasoning tasks. This work highlights retrieval and task-aware design as key factors for building practical and reproducible scholarly assistants.</abstract>
    <identifier type="citekey">kelber-etal-2026-need</identifier>
    <identifier type="doi">10.63317/2cuutd9tnnhp</identifier>
    <location>
        <url>https://aclanthology.org/2026.nslp-1.11/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>108</start>
            <end>118</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Do We Need Bigger Models for Science? Task-Aware Retrieval with Small Language Models
%A Kelber, Florian
%A Jobst, Matthias
%A Susanti, Yuni
%A Färber, Michael
%Y Rehm, Georg
%Y Dietze, Stefan
%Y Dessi, Danilo
%Y Maynard, Diana
%Y Schimmler, Sonja
%S Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026
%D 2026
%8 May
%I ELRA Language Resources Association (ELRA)
%C Palma, Mallorca (Spain)
%F kelber-etal-2026-need
%X Scientific knowledge discovery increasingly relies on large language models, yet many existing scholarly assistants depend on proprietary systems with tens or hundreds of billions of parameters. Such reliance limits reproducibility and accessibility for the research community. In this work, we ask a simple question: do we need bigger models for scientific applications? Specifically, we investigate to what extent carefully designed retrieval pipelines can compensate for reduced model scale in scientific applications. We design a lightweight retrieval-augmented framework that performs task-aware routing to select specialized retrieval strategies based on the input query. The system further integrates evidence from full-text scientific papers and structured scholarly metadata, and employs compact instruction-tuned language models to generate responses with citations. We evaluate the framework across several scholarly tasks, focusing on scholarly question answering (QA), including single- and multi-document scenarios, as well as biomedical QA under domain shift and scientific text compression. Our findings demonstrate that retrieval and model scale are complementary rather than interchangeable. While retrieval design can partially compensate for smaller models, model capacity remains important for complex reasoning tasks. This work highlights retrieval and task-aware design as key factors for building practical and reproducible scholarly assistants.
%R 10.63317/2cuutd9tnnhp
%U https://aclanthology.org/2026.nslp-1.11/
%U https://doi.org/10.63317/2cuutd9tnnhp
%P 108-118

Markdown (Informal)

[Do We Need Bigger Models for Science? Task-Aware Retrieval with Small Language Models](https://aclanthology.org/2026.nslp-1.11/) (Kelber et al., NSLP 2026)

• Do We Need Bigger Models for Science? Task-Aware Retrieval with Small Language Models (Kelber et al., NSLP 2026)

ACL

• Florian Kelber, Matthias Jobst, Yuni Susanti, and Michael Färber. 2026. Do We Need Bigger Models for Science? Task-Aware Retrieval with Small Language Models. In Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026, pages 108–118, Palma, Mallorca (Spain). ELRA Language Resources Association (ELRA).