Adapting large vision–language models for embedding tasks (VLM2Vec) is standard practice, but distilling them into compact architectures remains challenging due to capacity mismatches in tokenization, dimensionality, and multimodal representations. To address this, we propose a robust distillation framework based on two complementary mechanisms: Principal Geometry Alignment (PGA) to stabilize training by aligning principal geometric components, and a Mutual Information Maximization objective to enforce global semantic consistency across intra- and inter-modal pairs. By jointly constraining geometric structure and semantic dependency, our method effectively mitigates model capacity discrepancies. Experimental results on the Massive Multimodal Embedding Benchmark (MMEB) across all four meta-tasks—under both In-Distribution and Out-of-Distribution settings—demonstrate that our approach consistently outperforms state-of-the-art distillation methods. Our source code is available at https://github.com/theanh-ktmt/PGA-KD.