<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="shihab-etal-2026-spectral">
    <titleInfo>
        <title>Spectral DPO: Mitigating Validation Collapse via Heavy-Tailed Regularization</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Ibne</namePart>
        <namePart type="given">Farabi</namePart>
        <namePart type="family">Shihab</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sanjeda</namePart>
        <namePart type="family">Akter</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anuj</namePart>
        <namePart type="family">Sharma</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Direct Preference Optimization removes the explicit reward model of RLHF but inherits its central failure mode: the preference-reward proxy can keep improving while downstream reasoning, factuality, or safety degrades. We ask whether this reward overoptimization, which we label *validation collapse* as an operational name for the trajectory we measure, is visible in weight-space geometry. Drawing on Random Matrix Theory and the Heavy-Tailed Self-Regularisation hypothesis, we propose Spectral DPO, which augments the DPO objective with a stochastic penalty computed from a truncated top-k Lanczos estimate of the spectra of effective adapted projections. The penalty holds the fitted tail exponent near a heavy-tailed reference, caps scale-normalised growth of the largest singular direction, and matches the estimated spectrum to a heavy-tailed target. Across ten matched seeds, three model families, and five benchmarks spanning instruction following, dialogue, mathematical reasoning, factuality, and adversarial robustness, Spectral DPO gains 2.33–2.42 points over the strongest included baseline and sustains accuracy through ten epochs where vanilla DPO degrades. Controls and a post-hoc spectral repair rule out norm clipping, KL anchoring, and length inflation, and support the relevance of spectral geometry without identifying a single causal statistic. All runs use rank-64 LoRA, so the evidence concerns effective-spectrum control under rank-constrained adaptation.</abstract>
    <identifier type="citekey">shihab-etal-2026-spectral</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.785/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>18243</start>
            <end>18260</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Spectral DPO: Mitigating Validation Collapse via Heavy-Tailed Regularization
%A Shihab, Ibne Farabi
%A Akter, Sanjeda
%A Sharma, Anuj
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F shihab-etal-2026-spectral
%X Direct Preference Optimization removes the explicit reward model of RLHF but inherits its central failure mode: the preference-reward proxy can keep improving while downstream reasoning, factuality, or safety degrades. We ask whether this reward overoptimization, which we label *validation collapse* as an operational name for the trajectory we measure, is visible in weight-space geometry. Drawing on Random Matrix Theory and the Heavy-Tailed Self-Regularisation hypothesis, we propose Spectral DPO, which augments the DPO objective with a stochastic penalty computed from a truncated top-k Lanczos estimate of the spectra of effective adapted projections. The penalty holds the fitted tail exponent near a heavy-tailed reference, caps scale-normalised growth of the largest singular direction, and matches the estimated spectrum to a heavy-tailed target. Across ten matched seeds, three model families, and five benchmarks spanning instruction following, dialogue, mathematical reasoning, factuality, and adversarial robustness, Spectral DPO gains 2.33–2.42 points over the strongest included baseline and sustains accuracy through ten epochs where vanilla DPO degrades. Controls and a post-hoc spectral repair rule out norm clipping, KL anchoring, and length inflation, and support the relevance of spectral geometry without identifying a single causal statistic. All runs use rank-64 LoRA, so the evidence concerns effective-spectrum control under rank-constrained adaptation.
%U https://aclanthology.org/2026.emnlp-main.785/
%P 18243-18260

Markdown (Informal)

[Spectral DPO: Mitigating Validation Collapse via Heavy-Tailed Regularization](https://aclanthology.org/2026.emnlp-main.785/) (Shihab et al., EMNLP 2026)

• Spectral DPO: Mitigating Validation Collapse via Heavy-Tailed Regularization (Shihab et al., EMNLP 2026)

ACL

• Ibne Farabi Shihab, Sanjeda Akter, and Anuj Sharma. 2026. Spectral DPO: Mitigating Validation Collapse via Heavy-Tailed Regularization. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 18243–18260, Budapest, Hungary. Association for Computational Linguistics.