Recent work reports large gains for LLM agents on low-budget molecular optimization, with 2-3 times AUC improvements over reinforcement-learning and genetic-algorithm baselines on the PMO benchmark and similar gains on protein-targeted docking benchmarks. We investigate whether these gains reflect genuine optimization or benchmark artifacts, by re-evaluating a representative LLM agent under controlled blinding protocols and against a trivial random-walk baseline. We find that on PMO at the 50-call budget, the agent’s apparent advantage collapses below classical baselines once task names and curated reference compounds are removed from its prompt; a 2 times 2 ablation shows that retrieval from a small reference dictionary, not iterative optimization, is the dominant driver of the headline number. On a new protein-targeted lead-optimization benchmark with an AutoDock Vina composite oracle, a single-edit halogen/methyl random-walk mutator outperforms the LLM agent, a graph-based GA, and a non-iterative GPT baseline on every aggregate metric—a property of the oracle landscape under tight budgets, not of LLM agents per se. We release both protocols (name-blinded PMO scoring; random-walk lower bound for Vina-based protein benchmarks) as standard sanity checks for future LLM-agent papers in molecular optimization.