<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="wadhwa-etal-2025-taught">
    <titleInfo>
        <title>Who Taught You That? Tracing Teachers in Model Distillation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Somin</namePart>
        <namePart type="family">Wadhwa</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chantal</namePart>
        <namePart type="family">Shaib</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Silvio</namePart>
        <namePart type="family">Amir</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Byron</namePart>
        <namePart type="given">C</namePart>
        <namePart type="family">Wallace</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2025</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Wanxiang</namePart>
            <namePart type="family">Che</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Joyce</namePart>
            <namePart type="family">Nabende</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ekaterina</namePart>
            <namePart type="family">Shutova</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohammad</namePart>
            <namePart type="given">Taher</namePart>
            <namePart type="family">Pilehvar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Vienna, Austria</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-256-5</identifier>
    </relatedItem>
    <abstract>Model distillation – using outputs from a large teacher model to teach a small student model – is a practical means of creating efficient models for a particular task. We ask: Can we identify a students’ teacher based on its outputs? Such “footprints” left by teacher LLMs would be interesting artifacts. Beyond this, reliable teacher inference may have practical implications as actors seek to distill specific capabilities of massive proprietary LLMs into deployed smaller LMs, potentially violating terms of service. We consider practical task distillation targets including summarization, question answering, and instruction-following. We assume a finite set of candidate teacher models, which we treat as blackboxes. We design discriminative models that operate over lexical features. We find that n-gram similarity alone is unreliable for identifying teachers, but part-of-speech (PoS) templates preferred by student models mimic those of their teachers.</abstract>
    <identifier type="citekey">wadhwa-etal-2025-taught</identifier>
    <identifier type="doi">10.18653/v1/2025.findings-acl.173</identifier>
    <location>
        <url>https://aclanthology.org/2025.findings-acl.173/</url>
    </location>
    <part>
        <date>2025-07</date>
        <extent unit="page">
            <start>3307</start>
            <end>3315</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Who Taught You That? Tracing Teachers in Model Distillation
%A Wadhwa, Somin
%A Shaib, Chantal
%A Amir, Silvio
%A Wallace, Byron C.
%Y Che, Wanxiang
%Y Nabende, Joyce
%Y Shutova, Ekaterina
%Y Pilehvar, Mohammad Taher
%S Findings of the Association for Computational Linguistics: ACL 2025
%D 2025
%8 July
%I Association for Computational Linguistics
%C Vienna, Austria
%@ 979-8-89176-256-5
%F wadhwa-etal-2025-taught
%X Model distillation – using outputs from a large teacher model to teach a small student model – is a practical means of creating efficient models for a particular task. We ask: Can we identify a students’ teacher based on its outputs? Such “footprints” left by teacher LLMs would be interesting artifacts. Beyond this, reliable teacher inference may have practical implications as actors seek to distill specific capabilities of massive proprietary LLMs into deployed smaller LMs, potentially violating terms of service. We consider practical task distillation targets including summarization, question answering, and instruction-following. We assume a finite set of candidate teacher models, which we treat as blackboxes. We design discriminative models that operate over lexical features. We find that n-gram similarity alone is unreliable for identifying teachers, but part-of-speech (PoS) templates preferred by student models mimic those of their teachers.
%R 10.18653/v1/2025.findings-acl.173
%U https://aclanthology.org/2025.findings-acl.173/
%U https://doi.org/10.18653/v1/2025.findings-acl.173
%P 3307-3315

Markdown (Informal)

[Who Taught You That? Tracing Teachers in Model Distillation](https://aclanthology.org/2025.findings-acl.173/) (Wadhwa et al., Findings 2025)

• Who Taught You That? Tracing Teachers in Model Distillation (Wadhwa et al., Findings 2025)

ACL

• Somin Wadhwa, Chantal Shaib, Silvio Amir, and Byron C Wallace. 2025. Who Taught You That? Tracing Teachers in Model Distillation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 3307–3315, Vienna, Austria. Association for Computational Linguistics.