Soham Tripathy
Author directory2026
Confidence Drift During Task-Specific Language Model Fine-Tuning
Soham Tripathy
The Proceedings of the Third Workshop on Uncertainty-Aware NLP
Soham Tripathy
The Proceedings of the Third Workshop on Uncertainty-Aware NLP
Task-specific fine-tuning can change not only what a language model predicts, but also how confident it is in those predictions. We study confidence and accuracy drifts throughout LoRA fine-tuning of seven decoder-only language models spanning multiple families and scales. Models are fine-tuned on AG News and evaluated at intermediate checkpoints on the source task and five untouched cross-task benchmarks. We find that cross-task accuracy generally remains stable or degrades, while confidence increases on both correct and incorrect predictions. Across many settings, confidence follows a non-monotonic trajectory: after an early transient and a relatively stable middle stage, it rises again late in training, coinciding with increasing train–validation NLL divergence. Our results show that accuracy-only evaluation can miss confidence drifts introduced by task-specific fine-tuning.