<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="wang-etal-2026-learning-context">
    <titleInfo>
        <title>Learning-to-Context Slope: Evaluating In-Context Learning Effectiveness Beyond Performance Illusions</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Dingzirui</namePart>
        <namePart type="family">Wang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xuanliang</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Keyan</namePart>
        <namePart type="family">Xu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Qingfu</namePart>
        <namePart type="family">Zhu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Wanxiang</namePart>
        <namePart type="family">Che</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yang</namePart>
        <namePart type="family">Deng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>The effectiveness of in-context learning (ICL) varies significantly across models and tasks, posing challenges for practitioners to determine when ICL reliably improves performance. Current evaluation approaches, reliant on performance changes after applying ICL, suffer from low reliability, poor attribution, and impracticality in data-insufficient scenarios. We propose the Learning-to-Context Slope (LCS), a novel metric that quantifies ICL effectiveness by modeling the slope between learning gain (loss decrease from demonstrations) and contextual relevance (demonstration-input relevance). LCS addresses key limitations of performance-based metrics: (i) it captures loss changes even when outputs are incorrect; (ii) its formulation attributes ICL failures to weak contextual alignment or strong output calibration; (iii) it minimizes reliance on labeled data via synthetic evaluation. Experiments show that LCS can better reflect ICL effectiveness when performance change fails, demonstrating its effectiveness.</abstract>
    <identifier type="citekey">wang-etal-2026-learning-context</identifier>
    <identifier type="doi">10.1162/tacl.a.809</identifier>
    <location>
        <url>https://aclanthology.org/2026.tacl-1.112/</url>
    </location>
    <part>
        <date>2026</date>
        <detail type="volume"><number>14</number></detail>
        <extent unit="page">
            <start>2399</start>
            <end>2424</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Learning-to-Context Slope: Evaluating In-Context Learning Effectiveness Beyond Performance Illusions
%A Wang, Dingzirui
%A Zhang, Xuanliang
%A Xu, Keyan
%A Zhu, Qingfu
%A Che, Wanxiang
%A Deng, Yang
%J Transactions of the Association for Computational Linguistics
%D 2026
%V 14
%I MIT Press
%C Cambridge, MA
%F wang-etal-2026-learning-context
%X The effectiveness of in-context learning (ICL) varies significantly across models and tasks, posing challenges for practitioners to determine when ICL reliably improves performance. Current evaluation approaches, reliant on performance changes after applying ICL, suffer from low reliability, poor attribution, and impracticality in data-insufficient scenarios. We propose the Learning-to-Context Slope (LCS), a novel metric that quantifies ICL effectiveness by modeling the slope between learning gain (loss decrease from demonstrations) and contextual relevance (demonstration-input relevance). LCS addresses key limitations of performance-based metrics: (i) it captures loss changes even when outputs are incorrect; (ii) its formulation attributes ICL failures to weak contextual alignment or strong output calibration; (iii) it minimizes reliance on labeled data via synthetic evaluation. Experiments show that LCS can better reflect ICL effectiveness when performance change fails, demonstrating its effectiveness.
%R 10.1162/tacl.a.809
%U https://aclanthology.org/2026.tacl-1.112/
%U https://doi.org/10.1162/tacl.a.809
%P 2399-2424

Markdown (Informal)

[Learning-to-Context Slope: Evaluating In-Context Learning Effectiveness Beyond Performance Illusions](https://aclanthology.org/2026.tacl-1.112/) (Wang et al., TACL 2026)

• Learning-to-Context Slope: Evaluating In-Context Learning Effectiveness Beyond Performance Illusions (Wang et al., TACL 2026)

ACL

• Dingzirui Wang, Xuanliang Zhang, Keyan Xu, Qingfu Zhu, Wanxiang Che, and Yang Deng. 2026. Learning-to-Context Slope: Evaluating In-Context Learning Effectiveness Beyond Performance Illusions. Transactions of the Association for Computational Linguistics, 14:2399–2424.