Efficient training and adaptation are central to deploying language models under resource constraints. We present a controlled comparison of six parameter-efficient fine-tuning (PEFT) methods—LoRA, QLoRA, DoRA, PiSSA, LoftQ, and AdaLoRA—across the full cross-product of three base models (Qwen2.5-1.5B, Qwen2.5-7B, Llama-3-8B) and two mathematical reasoning benchmarks (GSM8K, MATH), with every configuration trained on MetaMathQA under one identical budget and reported as mean ± standard deviation over three seeds. Qwen2.5-1.5B + PiSSA reaches 71.40 ± 0.27% on GSM8K, exceeding the unadapted 7B base model (68.50 ± 0.42%; Welch’s t(3.4) = 10.06, p = 0.001). Significance testing shows that the method ordering—PiSSA > DoRA > LoRA ≈ AdaLoRA ≈ LoftQ ≈ QLoRA—is identical across all three base models (Kendall’s τ = 1.0), but that only the PiSSA and DoRA gains over LoRA survive multiple-comparison correction: the LoRA/QLoRA/LoftQ/AdaLoRA cluster is statistically indistinguishable at n=3. Four-bit QLoRA cuts peak training VRAM by up to 69.6% (72.8% with PagedAdamW), within one point of the 68.7% predicted by a static-memory model, while PagedAdamW trades 15–20% throughput for memory safety. We add rank and target-module ablations, a GSM8K error taxonomy, and the finding that PiSSA’s spectral initialization at r=16 outperforms LoRA at r=64, and we release the training, evaluation, and profiling code with the exact configuration for every run.