<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="tsymbalov-etal-2026-grace">
    <titleInfo>
        <title>GRACE-DS: a Guarded Reward-guided Agent Correction Environment in Data Science</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Aleksandr</namePart>
        <namePart type="family">Tsymbalov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Danis</namePart>
        <namePart type="family">Zaripov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Artem</namePart>
        <namePart type="family">Epifanov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anastasiya</namePart>
        <namePart type="family">Palienko</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Pepa</namePart>
            <namePart type="family">Atanasova</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Gholamreza</namePart>
            <namePart type="family">Haffari</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Minghan</namePart>
            <namePart type="family">Wang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-460-6</identifier>
    </relatedItem>
    <abstract>Evaluating LLM-powered AutoML agents for enterprise data science requires more than final leaderboard scores: agents must deliver reproducible, leakage-free, protocol-valid workflows on organization-specific data before deployment. We introduce GRACE-DS, a Guarded Reward-guided Agent Correction Environment in Data Science for evaluating tabular machine learning agents in real-world-like conditions. GRACE-DS stages the full workflow—planning, data inspection, feature engineering, modeling, validation, code repair, and final submission—inside an isolated sandbox where evaluator-owned labels and hidden tests remain private. Hidden executable validators track both predictive progress and process reliability, including leakage avoidance, reproducibility, validation discipline, correction behavior, reward alignment, and terminal submission validity. Based on 7,000 episodes spanning 8 LLMs, 10 tasks, 15 regimes, ablations, reward-optimization probes, and red-team settings, the structured flexible iterative regime (our approach) achieves the strongest end-to-end normalized hidden-test quality, outperforming single shot, unstructured agent, and restart baselines while improving protocol-valid completion. These results establish GRACE-DS as a controlled pre-deployment harness for diagnosing whether LLM-powered AutoML agents satisfy core enterprise execution constraints, on a deliberately search-free tabular sub-problem that isolates planning, correction, and validation discipline from expensive model search. An earlier version of this paper is available at https://arxiv.org/abs/2606.16000v2; code and artifacts are available at https://github.com/Alexx221x/GRACE-DS/tree/archive.</abstract>
    <identifier type="citekey">tsymbalov-etal-2026-grace</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-industry.244/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>4120</start>
            <end>4148</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T GRACE-DS: a Guarded Reward-guided Agent Correction Environment in Data Science
%A Tsymbalov, Aleksandr
%A Zaripov, Danis
%A Epifanov, Artem
%A Palienko, Anastasiya
%Y Atanasova, Pepa
%Y Haffari, Gholamreza
%Y Wang, Minghan
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-460-6
%F tsymbalov-etal-2026-grace
%X Evaluating LLM-powered AutoML agents for enterprise data science requires more than final leaderboard scores: agents must deliver reproducible, leakage-free, protocol-valid workflows on organization-specific data before deployment. We introduce GRACE-DS, a Guarded Reward-guided Agent Correction Environment in Data Science for evaluating tabular machine learning agents in real-world-like conditions. GRACE-DS stages the full workflow—planning, data inspection, feature engineering, modeling, validation, code repair, and final submission—inside an isolated sandbox where evaluator-owned labels and hidden tests remain private. Hidden executable validators track both predictive progress and process reliability, including leakage avoidance, reproducibility, validation discipline, correction behavior, reward alignment, and terminal submission validity. Based on 7,000 episodes spanning 8 LLMs, 10 tasks, 15 regimes, ablations, reward-optimization probes, and red-team settings, the structured flexible iterative regime (our approach) achieves the strongest end-to-end normalized hidden-test quality, outperforming single shot, unstructured agent, and restart baselines while improving protocol-valid completion. These results establish GRACE-DS as a controlled pre-deployment harness for diagnosing whether LLM-powered AutoML agents satisfy core enterprise execution constraints, on a deliberately search-free tabular sub-problem that isolates planning, correction, and validation discipline from expensive model search. An earlier version of this paper is available at https://arxiv.org/abs/2606.16000v2; code and artifacts are available at https://github.com/Alexx221x/GRACE-DS/tree/archive.
%U https://aclanthology.org/2026.emnlp-industry.244/
%P 4120-4148

Markdown (Informal)

[GRACE-DS: a Guarded Reward-guided Agent Correction Environment in Data Science](https://aclanthology.org/2026.emnlp-industry.244/) (Tsymbalov et al., EMNLP 2026)

• GRACE-DS: a Guarded Reward-guided Agent Correction Environment in Data Science (Tsymbalov et al., EMNLP 2026)

ACL

• Aleksandr Tsymbalov, Danis Zaripov, Artem Epifanov, and Anastasiya Palienko. 2026. GRACE-DS: a Guarded Reward-guided Agent Correction Environment in Data Science. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 4120–4148, Budapest, Hungary. Association for Computational Linguistics.