<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="shi-2026-gradient">
    <titleInfo>
        <title>Gradient-Aligned Iterative Fine-Tuning for Adaptive LLM Post-Training</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Zhan</namePart>
        <namePart type="family">Shi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yimu</namePart>
            <namePart type="family">Wang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yee</namePart>
            <namePart type="given">Man</namePart>
            <namePart type="family">Choi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Di</namePart>
            <namePart type="family">Wu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Siqi</namePart>
            <namePart type="family">Ouyang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mozhgan</namePart>
            <namePart type="given">Nasr</namePart>
            <namePart type="family">Azadani</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yi</namePart>
            <namePart type="given">R</namePart>
            <namePart type="given">(May)</namePart>
            <namePart type="family">Fung</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-478-1</identifier>
    </relatedItem>
    <abstract>Post-training has become the primary mechanism for grounding large language models (LLMs) to application-specific domains, enabling them to acquire new knowledge, behaviors, and policies beyond pre-training. Supervised fine-tuning (SFT) and preference optimization are the dominant paradigms for this adaptation, but are typically applied sequentially, requiring careful tuning of training schedules, regularization, and other hyperparameters.\par We propose Gradient-Aligned Iterative Fine-Tuning (GAIF), an adaptive post-training framework that combines three complementary components: a unified SFT+DPO objective optimized simultaneously; a prospective adaptive weighting mechanism that selects task weights based on next-step gradient geometry; and an iterative outer loop that regenerates preference pairs from the current model’s failures, keeping the supervision signal synchronized with the model’s evolving capabilities while adapting training toward arbitrary evaluation metrics, including non-differentiable ones.\par We evaluate GAIF on a production-scale generative movie search assistant, where a 7B LLM is adapted to the entertainment domain while remaining faithful to a fixed streaming catalog. The 7B model trained with GAIF outperforms standard SFT, sequential SFT+DPO, and a ~90B reference model in both ranking quality (NDCG) and output validity. Experiments on GSM8K mathematical reasoning further show the algorithm’s effectiveness extends beyond grounded retrieval.</abstract>
    <identifier type="citekey">shi-2026-gradient</identifier>
    <location>
        <url>https://aclanthology.org/2026.groundlm-1.36/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>453</start>
            <end>462</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Gradient-Aligned Iterative Fine-Tuning for Adaptive LLM Post-Training
%A Shi, Zhan
%Y Wang, Yimu
%Y Choi, Yee Man
%Y Wu, Di
%Y Ouyang, Siqi
%Y Azadani, Mozhgan Nasr
%Y Fung, Yi R. (May)
%S Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-478-1
%F shi-2026-gradient
%X Post-training has become the primary mechanism for grounding large language models (LLMs) to application-specific domains, enabling them to acquire new knowledge, behaviors, and policies beyond pre-training. Supervised fine-tuning (SFT) and preference optimization are the dominant paradigms for this adaptation, but are typically applied sequentially, requiring careful tuning of training schedules, regularization, and other hyperparameters.\par We propose Gradient-Aligned Iterative Fine-Tuning (GAIF), an adaptive post-training framework that combines three complementary components: a unified SFT+DPO objective optimized simultaneously; a prospective adaptive weighting mechanism that selects task weights based on next-step gradient geometry; and an iterative outer loop that regenerates preference pairs from the current model’s failures, keeping the supervision signal synchronized with the model’s evolving capabilities while adapting training toward arbitrary evaluation metrics, including non-differentiable ones.\par We evaluate GAIF on a production-scale generative movie search assistant, where a 7B LLM is adapted to the entertainment domain while remaining faithful to a fixed streaming catalog. The 7B model trained with GAIF outperforms standard SFT, sequential SFT+DPO, and a ~90B reference model in both ranking quality (NDCG) and output validity. Experiments on GSM8K mathematical reasoning further show the algorithm’s effectiveness extends beyond grounded retrieval.
%U https://aclanthology.org/2026.groundlm-1.36/
%P 453-462

Markdown (Informal)

[Gradient-Aligned Iterative Fine-Tuning for Adaptive LLM Post-Training](https://aclanthology.org/2026.groundlm-1.36/) (Shi, GroundLM 2026)

• Gradient-Aligned Iterative Fine-Tuning for Adaptive LLM Post-Training (Shi, GroundLM 2026)

ACL

• Zhan Shi. 2026. Gradient-Aligned Iterative Fine-Tuning for Adaptive LLM Post-Training. In Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026), pages 453–462, Budapest, Hungary. Association for Computational Linguistics.