Zhuofan Ying

Author directory

2026

Linear probes are widely used in interpretability research and often compared by cosine similarity. The Mahalanobis cosine similarity (MCS) between two directions, which reweights the inner product by test data covariance, is a natural task-aware refinement. Ying et al. (2026) report a nearly perfect linear relationship between a probe’s MCS to a reference probe trained on out-of-distribution (OOD) data and its OOD AUROC (R^2=0.98). Here, we extend this empirical finding across models, layers, and concept domains, and prove it in closed form: For balanced classes whose projections are Gaussian, OOD AUROC and MCS to the reference probe are near-linear because both are sigmoid-shaped functions of the probe’s signal-to-noise ratio (SNR) on the test data. The theory also predicts when this linearity fails, which we verify empirically. MCS offers a theoretically grounded and empirically effective alternative to Euclidean cosine similarity for comparing linear probes.