<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="sundrani-etal-2026-low">
    <titleInfo>
        <title>Low-Rank Compression of Language Models via Differentiable Rank Selection</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sidhant</namePart>
        <namePart type="family">Sundrani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Francesco</namePart>
        <namePart type="family">Tudisco</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Pasquale</namePart>
        <namePart type="family">Minervini</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Fifteenth Language Resources and Evaluation Conference</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Stelios</namePart>
            <namePart type="family">Piperidis</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Núria</namePart>
            <namePart type="family">Bel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Henk</namePart>
            <namePart type="family">van den Heuvel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nancy</namePart>
            <namePart type="family">Ide</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Simon</namePart>
            <namePart type="family">Krek</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Antonio</namePart>
            <namePart type="family">Toral</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resource Association</publisher>
            <place>
                <placeTerm type="text">Palma de Mallorca, Spain</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Approaches for compressing large-language models using low-rank decomposition have made strides, particularly with the introduction of activation and loss-aware SVD, which improves the trade-off between decomposition rank and downstream task performance. Despite these advancements, a persistent challenge remains–selecting the optimal ranks for each layer to jointly optimise compression rate and downstream task accuracy. Current methods either rely on heuristics that can yield sub-optimal results due to their limited discrete search space or are gradient-based but are not as performant as heuristic approaches without post-compression fine-tuning. To address these issues, we propose Learning to Low-Rank Compress (LLRC), a gradient-based approach that directly learns the weights of masks that select singular values in a fine-tuning-free setting. Using a calibration dataset, we train only the mask weights to select fewer and fewer singular values while minimising the divergence of intermediate activations from the original model. Our approach outperforms competing methods that similarly require no post-compression fine-tuning across various compression rates on common-sense reasoning and open-domain question-answering tasks. For instance, with a compression rate of 20% on Llama-2-13B, LLRC outperforms the competitive Sensitivity-based Truncation Rank Searching (STRS) on MMLU, BoolQ, and OpenbookQA by 12%, 3.5%, and 4.4%, respectively. Compared to other compression techniques, our approach consistently outperforms fine-tuning-free variants of SVD-LLM and LLM-Pruner across datasets and compression rates. Our approach also performs competitively with LLM-Pruner after fine-tuning on Llama-2-7B and Llama-2-13B.</abstract>
    <identifier type="citekey">sundrani-etal-2026-low</identifier>
    <identifier type="doi">10.63317/2xbs948bhby9</identifier>
    <location>
        <url>https://aclanthology.org/2026.lrec-1.787/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>10031</start>
            <end>10045</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Low-Rank Compression of Language Models via Differentiable Rank Selection
%A Sundrani, Sidhant
%A Tudisco, Francesco
%A Minervini, Pasquale
%Y Piperidis, Stelios
%Y Bel, Núria
%Y van den Heuvel, Henk
%Y Ide, Nancy
%Y Krek, Simon
%Y Toral, Antonio
%S Proceedings of the Fifteenth Language Resources and Evaluation Conference
%D 2026
%8 May
%I ELRA Language Resource Association
%C Palma de Mallorca, Spain
%F sundrani-etal-2026-low
%X Approaches for compressing large-language models using low-rank decomposition have made strides, particularly with the introduction of activation and loss-aware SVD, which improves the trade-off between decomposition rank and downstream task performance. Despite these advancements, a persistent challenge remains–selecting the optimal ranks for each layer to jointly optimise compression rate and downstream task accuracy. Current methods either rely on heuristics that can yield sub-optimal results due to their limited discrete search space or are gradient-based but are not as performant as heuristic approaches without post-compression fine-tuning. To address these issues, we propose Learning to Low-Rank Compress (LLRC), a gradient-based approach that directly learns the weights of masks that select singular values in a fine-tuning-free setting. Using a calibration dataset, we train only the mask weights to select fewer and fewer singular values while minimising the divergence of intermediate activations from the original model. Our approach outperforms competing methods that similarly require no post-compression fine-tuning across various compression rates on common-sense reasoning and open-domain question-answering tasks. For instance, with a compression rate of 20% on Llama-2-13B, LLRC outperforms the competitive Sensitivity-based Truncation Rank Searching (STRS) on MMLU, BoolQ, and OpenbookQA by 12%, 3.5%, and 4.4%, respectively. Compared to other compression techniques, our approach consistently outperforms fine-tuning-free variants of SVD-LLM and LLM-Pruner across datasets and compression rates. Our approach also performs competitively with LLM-Pruner after fine-tuning on Llama-2-7B and Llama-2-13B.
%R 10.63317/2xbs948bhby9
%U https://aclanthology.org/2026.lrec-1.787/
%U https://doi.org/10.63317/2xbs948bhby9
%P 10031-10045

Markdown (Informal)

[Low-Rank Compression of Language Models via Differentiable Rank Selection](https://aclanthology.org/2026.lrec-1.787/) (Sundrani et al., LREC 2026)

• Low-Rank Compression of Language Models via Differentiable Rank Selection (Sundrani et al., LREC 2026)

ACL

• Sidhant Sundrani, Francesco Tudisco, and Pasquale Minervini. 2026. Low-Rank Compression of Language Models via Differentiable Rank Selection. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 10031–10045, Palma de Mallorca, Spain. ELRA Language Resource Association.