Reasoning alignment usually assumes each query has a determinate answer, rewarding models for reaching it and penalizing or ignoring failures. This assumption breaks on flawed queries with missing information or contradictory constraints, causing models to hallucinate invalid answers or over-refuse difficult valid tasks. We study this failure mode as solvability boundary learning: separating input-side answerability defects from model-side inability on well-posed tasks. To address this, we introduce UnsolvableQA, a verifier-backed multi-domain benchmark built with deterministic checks and audited math verification, and UnsolvableRL, an alignment framework that decouples reasoning, unsolvability detection, and capability-calibrated refusal. UnsolvableRL yields over 85% unsolvability detection on Qwen3-4B while improving solvable accuracy from 43.4% to 69.4%, with a +35.4-point solvable-accuracy gain on Llama3-8B. Diagnostic tuning further distinguishes missing-information defects from constraint conflicts and transfers to interactive clarification, improving Detail Recovery from 47.8% to 60.7% for Llama3-8B. Ablations reveal Capability Collapse: refusal penalties degrade detection without unsolvable data, but act as a rigor regularizer when such data are included.

Large Language Models (LLMs) have achieved remarkable success across diverse tasks, largely driven by well-designed prompts. However, crafting and selecting such prompts often requires considerable human effort, significantly limiting its scalability. To mitigate this, recent studies have explored automated prompt optimization as a promising solution. Despite these efforts, existing methods still face critical challenges in robustness, efficiency, and generalization. To systematically address these challenges, we first conduct an empirical analysis to identify the limitations of current reflection-based prompt optimization paradigm. Building on these insights, we propose 7 innovative approaches inspired by traditional deep learning paradigms for prompt optimization (DLPO), seamlessly integrating these concepts into text-based gradient optimization. Through these advancements, we progressively tackle the aforementioned challenges and validate our methods through extensive experimentation. We hope our study not only provides valuable guidance for future research but also offers a comprehensive understanding of the challenges and potential solutions in prompt optimization.