<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="gowda-2026-quantize">
    <titleInfo>
        <title>Quantize, Qualify, Rerank: A Recipe for Compressing LLMs Without Losing Quality</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Thamme</namePart>
        <namePart type="family">Gowda</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Eleventh Conference on Machine Translation</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Barry</namePart>
            <namePart type="family">Haddow</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tom</namePart>
            <namePart type="family">Kocmi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Philipp</namePart>
            <namePart type="family">Koehn</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Christof</namePart>
            <namePart type="family">Monz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-474-3</identifier>
    </relatedItem>
    <abstract>Compressing large language models lowers serving cost, but a single automatic metric may miss quality loss. For the WMT26 Model Compression constrained track, we compress ‘google/gemma-3-12b-it‘ (24.4 GB in bf16) for three translation directions on one H100. Paired bootstrap analysis exposes metric disagreement on 4-bit quantization; guided by WMT24 human meta-evaluation, we use MetricX24-XXL as primary and CometKiwi-XXL as a cross-check. Removing the unused vision stack and non-task vocabulary and using int8 weights or fp8 activations causes no measurable loss, whereas int4 leaves a small residual. QLoRA healing overfits its calibration domain. A cheap Cometoid best-of-4 reranker at ‘T=0.3‘ improves both the 8-bit and 4-bit systems under held-out CometKiwi-XXL and reference-based MetricX24-XXL. Both surpass greedy bf16 on CometKiwi-XXL; the 8-bit system also surpasses it on MetricX24-XXL, while the 4-bit system nearly matches it.</abstract>
    <identifier type="citekey">gowda-2026-quantize</identifier>
    <location>
        <url>https://aclanthology.org/2026.wmt-main.136/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>1991</start>
            <end>1998</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Quantize, Qualify, Rerank: A Recipe for Compressing LLMs Without Losing Quality
%A Gowda, Thamme
%Y Haddow, Barry
%Y Kocmi, Tom
%Y Koehn, Philipp
%Y Monz, Christof
%S Proceedings of the Eleventh Conference on Machine Translation
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-474-3
%F gowda-2026-quantize
%X Compressing large language models lowers serving cost, but a single automatic metric may miss quality loss. For the WMT26 Model Compression constrained track, we compress ‘google/gemma-3-12b-it‘ (24.4 GB in bf16) for three translation directions on one H100. Paired bootstrap analysis exposes metric disagreement on 4-bit quantization; guided by WMT24 human meta-evaluation, we use MetricX24-XXL as primary and CometKiwi-XXL as a cross-check. Removing the unused vision stack and non-task vocabulary and using int8 weights or fp8 activations causes no measurable loss, whereas int4 leaves a small residual. QLoRA healing overfits its calibration domain. A cheap Cometoid best-of-4 reranker at ‘T=0.3‘ improves both the 8-bit and 4-bit systems under held-out CometKiwi-XXL and reference-based MetricX24-XXL. Both surpass greedy bf16 on CometKiwi-XXL; the 8-bit system also surpasses it on MetricX24-XXL, while the 4-bit system nearly matches it.
%U https://aclanthology.org/2026.wmt-main.136/
%P 1991-1998

Markdown (Informal)

[Quantize, Qualify, Rerank: A Recipe for Compressing LLMs Without Losing Quality](https://aclanthology.org/2026.wmt-main.136/) (Gowda, WMT 2026)

• Quantize, Qualify, Rerank: A Recipe for Compressing LLMs Without Losing Quality (Gowda, WMT 2026)

ACL

• Thamme Gowda. 2026. Quantize, Qualify, Rerank: A Recipe for Compressing LLMs Without Losing Quality. In Proceedings of the Eleventh Conference on Machine Translation, pages 1991–1998, Budapest, Hungary. Association for Computational Linguistics.