Small language models are attractive in production because they are inexpensive, fast, and easy to specialize, but specialization remains a difficult loop. The hard decisions surround training: acquiring and curating supervision, diagnosing failures, selecting a training strategy, and preventing a targeted fix from breaking behavior that already works. We present Pioneer Agent, a closed-loop system that automates this lifecycle. In cold-start mode, the agent turns a task specification into a trained model by searching jointly over data composition, hyperparameters, and learning strategy. In production mode, it converts judged failures into a failure taxonomy and corrective curriculum, retrains, and retains an update only when it passes regression gates. We introduce AdaptFT-Bench, seven staged scenarios that combine fixable noise with harmful supervision at increasing poison rates. Across eight cold-start benchmarks, Pioneer Agent improves over the base checkpoint on every task. Across all seven AdaptFT-Bench scenarios, it improves or preserves the deployed checkpoint while indiscriminate retraining degrades, with final gaps up to 43 points. In two production-style case studies, the same loop raises intent-classification pass rate from 84.9% to 99.3% and entity F1 from 0.345 to 0.810. These results show that deployment failures can be converted into effective training curricula without manual intervention between iterations.

Modern causal language models stack many attention blocks to improve performance, but not all blocks are necessary for every task. We propose Hopscotch, a simple yet effective method that identifies and skips attention blocks with least contributions to a task and adapts to preserve output quality. Hopscotch jointly optimizes which blocks to skip and how to scale the outputs of the remaining layers. By introducing lightweight, trainable scaling parameters to attention and MLP blocks, it mitigates distribution shifts in hidden states caused by removing attention blocks. Hopscotch does not modify model weights or require access to pretraining or instruction-tuning data, and is compatible with existing model compression techniques. When applied to Llama-3.1-8B and Qwen-2.5-7B, Hopscotch achieves less than a 2% drop in performance even after skipping four attention blocks.