Post-training quantization methods universally minimize element-wise reconstruction error, yet we show this is the wrong objective. A systematic correlation study on LLaMA models reveals that the representation kernel (the matrix of pairwise cosine similarities among calibration inputs) is a substantially stronger proxy for downstream task performance than weight MSE or output MSE. Building on this finding, we propose KernelQuant, a training-free PTQ framework comprising: KPQ, which selects weight-quantization scales via kernel-MSE grid search followed by least-squares error compensation; model-level Hadamard rotation; and KPC, a kernel-preserving per-channel activation clipping scheme. KernelQuant introduces zero inference overhead beyond QuaRot and calibrates in minutes rather than hours. On LLaMA 3B–70B, KernelQuant achieves state-of-the-art W4A16 perplexity and competitive W4A4 results. On reasoning benchmarks (MMLU), where 4-bit quantization causes a “reasoning cliff,” KernelQuant retains the highest fraction of FP16 accuracy among all evaluated baselines, both rotation-based (QuaRot, SpinQuant, FlatQuant) and element-wise, reversing the perplexity-based ranking and supporting the view that reasoning depends on the geometric relations the kernel preserves.