Abhishek Upperwal
Author directory2026
Task–Specificity Score: Measuring How Much Instructions Really Matter for Supervision
Pritam Kadasi | Abhishek Upperwal | Mayank Singh
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Pritam Kadasi | Abhishek Upperwal | Mayank Singh
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Instruction tuning is now the default way to train and adapt large language models, but many instruction–input–output pairs are only weakly specified: for a given input, the same output can remain plausible under several alternative instructions. This raises a simple question: does the instruction uniquely determine the target output? We propose the Task–Specificity Score (TSS) to quantify how much an instruction matters for predicting the target output by contrasting the true instruction against plausible alternatives for the same input. We further introduce TSS++, which uses hard alternatives and a small quality term to mitigate easy-negative effects. Across three instruction datasets (Alpaca, Dolly-15K, NI-20) and three open LLMs (Gemma, Llama, Qwen), our strongest evidence appears in the tightest budget regime, particularly at 5% retention. In this setting, TSS and TSS++ consistently outperform existing selection baselines across most model–dataset combinations. These results show that task specificity is a useful signal for low-budget instruction-data selection and complements existing filters such as perplexity and IFD.