Gabriel Dershowitz

Author directory

2026

Controlled fine-tuning of pretrained text encoders — via gradual layer-wise unfreezing and anchoring — is widely assumed to benefit downstream tasks. We test this assumption in two-tower recommendation and findthree negative results. First, any fine-tuning dramatically beats a frozen encoder (≥ 2.7×cold Recall@10 lift); freezing the item tower is strongly counterproductive. Second, gradual unfreezing worsens performance compared to vanilla fine-tuning under realistic budgets (12epochs). Third, a fixed cosine drift penalty is the only consistently beneficial component; our full adaptive recipe does not outperform vanilla fine-tuning. Thus, for two-tower retrieval, practitioners should default to vanilla fine-tuning with light regularization — gradual unfreezing is unnecessary and harmful, challenging conventional wisdom from language model fine-tuning.