Adapt the Words, Freeze the Model: LLM Prompt Evolution for Domain-Shift-Robust Segmentation

Olga Volkova, Nikita Glazkov, Pchelin Konstantin, Iaroslav Radionovich Bespalov, Evgeny Evgenievich Baraboshkin, Denis Orlov, Dmitry V. Dylov


Abstract
In specialist industrial imagery the dominant cost is maintenance under domain shift: labeling and fine-tuning must be redone on every new field, and proprietary imagery often cannot leave the operator’s environment. For geological drill-core fracture segmentation we release CoreFrac, to our knowledge the first open pixel-level benchmark on drill-core photographs: 200 expert-annotated full-resolution images with an expert-agreement ceiling (soft-F1 0.86). We retarget a frozen concept segmentor (SAM 3) by evolving only its natural-language prompt — no gradient updates, just a small labeled probe. Under a source-grouped protocol the evolved prompt reaches 0.545 positives-only soft-F1, beating rank-8 LoRA at the same 96-patch cap (0.487; paired +0.058, 95% CI [+0.020, +0.094]) and the best of eight expert prompts by 2.7×. Under a controlled domain shift, re-evolving on a small probe lifts a 0.622 → 0.409 drop back to 0.534. Its masks distill into a small on-premises U-Net (7.8M parameters, 3.8 ms/patch) that retains ≈92% of the teacher’s positives-only soft-F1 (0.501) at ≈1700× lower latency than *unbatched* SAM 3 — a cheaper, leak-free deployment. Code and data are released.
Anthology ID:
2026.emnlp-industry.214
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
3588–3601
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.214/
DOI:
Bibkey:
Cite (ACL):
Olga Volkova, Nikita Glazkov, Pchelin Konstantin, Iaroslav Radionovich Bespalov, Evgeny Evgenievich Baraboshkin, Denis Orlov, and Dmitry V. Dylov. 2026. Adapt the Words, Freeze the Model: LLM Prompt Evolution for Domain-Shift-Robust Segmentation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 3588–3601, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Adapt the Words, Freeze the Model: LLM Prompt Evolution for Domain-Shift-Robust Segmentation (Volkova et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.214.pdf