Ziyuan Wang

Author directory

Other people with similar names: Ziyuan Wang

Unverified author pages with similar names: Ziyuan Wang


2026

Contrastive learning is widely used for sentence embedding training, yet it provides limited control over the global geometry of the learned embedding space. Sentence embeddings often exhibit spectral concentration, where a small set of directions dominates the representation while many others remain weakly active. We present singular value regularization (SVR), a batch-level framework for shaping sentence embedding spaces by directly controlling the singular value spectrum during training. SVR enables explicit control over both the spectral floor and the number of regularized directions. Experiments show consistent improvements across encoder backbones, pooling strategies, English evaluation tasks, and multilingual settings. Our analysis further shows that the best regularization range is not full-dimensional flattening, but a controlled active-dimensional region aligned with the naturally active subspace of pretrained representations. Moreover, the strongest results are obtained with a large SVR weight, indicating that spectral shaping can provide a substantial optimization signal alongside contrastive supervision. These findings motivate a geometry-aware perspective on sentence embedding learning, in which semantic supervision and explicit control of the embedding-space geometry play complementary roles.