In specialist industrial imagery the dominant cost is maintenance under domain shift: labeling and fine-tuning must be redone on every new field, and proprietary imagery often cannot leave the operator’s environment. For geological drill-core fracture segmentation we release CoreFrac, to our knowledge the first open pixel-level benchmark on drill-core photographs: 200 expert-annotated full-resolution images with an expert-agreement ceiling (soft-F1 0.86). We retarget a frozen concept segmentor (SAM 3) by evolving only its natural-language prompt — no gradient updates, just a small labeled probe. Under a source-grouped protocol the evolved prompt reaches 0.545 positives-only soft-F1, beating rank-8 LoRA at the same 96-patch cap (0.487; paired +0.058, 95% CI [+0.020, +0.094]) and the best of eight expert prompts by 2.7×. Under a controlled domain shift, re-evolving on a small probe lifts a 0.622 → 0.409 drop back to 0.534. Its masks distill into a small on-premises U-Net (7.8M parameters, 3.8 ms/patch) that retains ≈92% of the teacher’s positives-only soft-F1 (0.501) at ≈1700× lower latency than *unbatched* SAM 3 — a cheaper, leak-free deployment. Code and data are released.