<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="gong-etal-2026-dseffi">
    <titleInfo>
        <title>DSEffi-Bench: Demystifying Large Language Models’ Capability in Efficient Data Science Code Generation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Zhihao</namePart>
        <namePart type="family">Gong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Junzhe</namePart>
        <namePart type="family">Yu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dong</namePart>
        <namePart type="family">Huang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Zeyu</namePart>
        <namePart type="family">Sun</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jie</namePart>
        <namePart type="given">M</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dan</namePart>
        <namePart type="family">Hao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Current data science (DS) code generation benchmarks equate correctness with quality, overlooking execution time differences that span orders of magnitude between correct solutions. We introduce DSEffi-Bench, the first benchmark specifically targeting execution efficiency in LLM-generated DS code, comprising 1,000 instances across 10+ DS libraries with stress-testing harnesses and human-validated references. Evaluating 16 models across 3 tiers, we find that correctness alone fails to characterize efficiency: GPT-5.4 leads in correctness (Pass, 66.9%) but its efficiency score (B|P, 71.7%) nearly matches GPT-5.4-mini (71.6%), which solves 47 fewer tasks; Kimi-K2.5 ranks lowest in correctness among frontier models (40.2%) yet achieves the highest efficiency score (73.6%) across all 16 models. A human-annotated five-category taxonomy reveals that 79.1% of efficiency deficits extend beyond algorithmic complexity to domain-specific root causes, with distinct failure profiles across model tiers and libraries. Two exploratory experiments provide initial evidence that these diagnostics can guide improvement, yielding up to +14.7% efficiency gains via taxonomy-guided optimization and approaching Claude-Opus-4.6 Best@3 in efficiency at 13.0× lower cost via library-conditioned routing. Code and data are available at https://github.com/Albert-Gong/DSEffi-Bench.</abstract>
    <identifier type="citekey">gong-etal-2026-dseffi</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.318/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>7337</start>
            <end>7372</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T DSEffi-Bench: Demystifying Large Language Models’ Capability in Efficient Data Science Code Generation
%A Gong, Zhihao
%A Yu, Junzhe
%A Huang, Dong
%A Sun, Zeyu
%A Zhang, Jie M.
%A Hao, Dan
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F gong-etal-2026-dseffi
%X Current data science (DS) code generation benchmarks equate correctness with quality, overlooking execution time differences that span orders of magnitude between correct solutions. We introduce DSEffi-Bench, the first benchmark specifically targeting execution efficiency in LLM-generated DS code, comprising 1,000 instances across 10+ DS libraries with stress-testing harnesses and human-validated references. Evaluating 16 models across 3 tiers, we find that correctness alone fails to characterize efficiency: GPT-5.4 leads in correctness (Pass, 66.9%) but its efficiency score (B|P, 71.7%) nearly matches GPT-5.4-mini (71.6%), which solves 47 fewer tasks; Kimi-K2.5 ranks lowest in correctness among frontier models (40.2%) yet achieves the highest efficiency score (73.6%) across all 16 models. A human-annotated five-category taxonomy reveals that 79.1% of efficiency deficits extend beyond algorithmic complexity to domain-specific root causes, with distinct failure profiles across model tiers and libraries. Two exploratory experiments provide initial evidence that these diagnostics can guide improvement, yielding up to +14.7% efficiency gains via taxonomy-guided optimization and approaching Claude-Opus-4.6 Best@3 in efficiency at 13.0× lower cost via library-conditioned routing. Code and data are available at https://github.com/Albert-Gong/DSEffi-Bench.
%U https://aclanthology.org/2026.emnlp-main.318/
%P 7337-7372

Markdown (Informal)

[DSEffi-Bench: Demystifying Large Language Models’ Capability in Efficient Data Science Code Generation](https://aclanthology.org/2026.emnlp-main.318/) (Gong et al., EMNLP 2026)

• DSEffi-Bench: Demystifying Large Language Models’ Capability in Efficient Data Science Code Generation (Gong et al., EMNLP 2026)

ACL

• Zhihao Gong, Junzhe Yu, Dong Huang, Zeyu Sun, Jie M. Zhang, and Dan Hao. 2026. DSEffi-Bench: Demystifying Large Language Models’ Capability in Efficient Data Science Code Generation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 7337–7372, Budapest, Hungary. Association for Computational Linguistics.