Aleksandr Tsymbalov
Author directory2026
GRACE-DS: a Guarded Reward-guided Agent Correction Environment in Data Science
Aleksandr Tsymbalov | Danis Zaripov | Artem Epifanov | Anastasiya Palienko
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Aleksandr Tsymbalov | Danis Zaripov | Artem Epifanov | Anastasiya Palienko
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Evaluating LLM-powered AutoML agents for enterprise data science requires more than final leaderboard scores: agents must deliver reproducible, leakage-free, protocol-valid workflows on organization-specific data before deployment. We introduce GRACE-DS, a Guarded Reward-guided Agent Correction Environment in Data Science for evaluating tabular machine learning agents in real-world-like conditions. GRACE-DS stages the full workflow—planning, data inspection, feature engineering, modeling, validation, code repair, and final submission—inside an isolated sandbox where evaluator-owned labels and hidden tests remain private. Hidden executable validators track both predictive progress and process reliability, including leakage avoidance, reproducibility, validation discipline, correction behavior, reward alignment, and terminal submission validity. Based on 7,000 episodes spanning 8 LLMs, 10 tasks, 15 regimes, ablations, reward-optimization probes, and red-team settings, the structured flexible iterative regime (our approach) achieves the strongest end-to-end normalized hidden-test quality, outperforming single shot, unstructured agent, and restart baselines while improving protocol-valid completion. These results establish GRACE-DS as a controlled pre-deployment harness for diagnosing whether LLM-powered AutoML agents satisfy core enterprise execution constraints, on a deliberately search-free tabular sub-problem that isolates planning, correction, and validation discipline from expensive model search. An earlier version of this paper is available at https://arxiv.org/abs/2606.16000v2; code and artifacts are available at https://github.com/Alexx221x/GRACE-DS/tree/archive.