Mixed-precision quantization is a promising approach for compressing large language models under tight memory budgets. However, existing mixed-precision methods typically suffer from one of two limitations: they either rely on expensive discrete optimization to determine precision allocation, or introduce hardware inefficiencies due to irregular memory layouts. We propose SFMP, a solver-free and hardware-friendly mixed-precision quantization framework for large language models. SFMP defines a theoretically simple objective for mixed-precision quantization and is built upon three novel ideas: 1) Block-wise mixed-precision, enabling fine-grained precision within weight matrices while remaining hardware-friendly; 2) Row-column weight reordering, which improves structural alignment via row and column reordering, incurring only a small activation reordering overhead during inference; 3) Unified GEMM kernel, which supports block-wise mixed-precision GEMM at arbitrary average bit-widths. Extensive experiments demonstrate that SFMP outperforms state-of-the-art layer-wise mixed-precision methods under the same memory constraints, while significantly reducing quantization cost and improving inference efficiency. Code is available at https://github.com/Nkniexin/SFMP.