Eshita Jha

Author directory

2026

Deploying large language models for specialized tasks requires lengthy prompts combining schema, demonstrations, and instructions — creating a fundamental tension between prompt richness and inference feasibility. We identify and characterize two distinct prompt regimes: pattern-dominant tasks, where demonstrations drive performance through implicit formatting and extraction conventions, and procedure-dominant tasks, where explicit instructions carry the critical signal. To surface this distinction, we design a two-inference diagnostic that compares model performance with instructions alone versus examples alone, revealing which component to target for compression. Guided by this diagnostic, we introduce PRISM, a self-distillation framework that generates silver training data from the model’s own full-prompt outputs and trains lightweight LoRA adapters to internalize the dominant component’s behavior. PRISM recovers 131% of example benefit on CUAD and 151% on MultiWOZ (pattern-dominant), and 39% of instruction benefit on GSM8K (procedure-dominant), eliminating 72–79% of prompt tokens at inference time. Silver data captures 80–94% of gold-labeled performance without human annotations. Beyond compression, PRISM frees context capacity: replacing a long prompt with LoRA weights enables 89.5% compliance with new runtime instructions versus 75.4% with recency prompting. The diagnostic suggests a practical pre-training heuristic to indicate when distillation is likely to succeed, helping practitioners avoid costly training on non-beneficial components.