Reasoning alignment usually assumes each query has a determinate answer, rewarding models for reaching it and penalizing or ignoring failures. This assumption breaks on flawed queries with missing information or contradictory constraints, causing models to hallucinate invalid answers or over-refuse difficult valid tasks. We study this failure mode as solvability boundary learning: separating input-side answerability defects from model-side inability on well-posed tasks. To address this, we introduce UnsolvableQA, a verifier-backed multi-domain benchmark built with deterministic checks and audited math verification, and UnsolvableRL, an alignment framework that decouples reasoning, unsolvability detection, and capability-calibrated refusal. UnsolvableRL yields over 85% unsolvability detection on Qwen3-4B while improving solvable accuracy from 43.4% to 69.4%, with a +35.4-point solvable-accuracy gain on Llama3-8B. Diagnostic tuning further distinguishes missing-information defects from constraint conflicts and transfers to interactive clarification, improving Detail Recovery from 47.8% to 60.7% for Llama3-8B. Ablations reveal Capability Collapse: refusal penalties degrade detection without unsolvable data, but act as a rigor regularizer when such data are included.