Large language models (LLMs) exhibit strong English performance due to its dominance in pretraining data, yet remain inconsistent across non-English languages. Existing cross-lingual alignment methods rely on contrastive objectives that symmetrically perturb English representations, inadvertently degrading the proficiency they use as a reference. We propose an Optimal Transport (OT) framework that maps non-English hidden-state distributions toward English at middle transformer layers via entropy-regularised Sinkhorn iterations, while an English consistency regularisation term keeps the alignment anchor stationary. This asymmetric formulation preserves English performance by construction. Experiments on XNLI, XQuAD, and MMMLU show that our method delivers strong cross-lingual transfer while successfully avoiding the English performance degradation observed in the contrastive baseline.

Few-shot Continual Relation Extraction (FCRE) has emerged as a significant challenge in information extraction, necessitating that relation extraction (RE) systems can sequentially identify new relations with limited labeled samples. While existing studies have demonstrated promising results in FCRE, they often overlook the issue of similar relations, which is a critical factor contributing to catastrophic forgetting. In this work, we propose Sirus–a novel method that utilizes relation descriptions and dynamic clustering on these descriptions to identify similar relations. Leveraging this information, we introduce innovative loss functions specifically designed to enhance the distinction between relations, with a focus on learning to differentiate similar ones. Experimental results show that our approach can effectively mitigate the problem of catastrophic forgetting and outperforms state-of-the-art methods by a large margin. Additionally, we explore the potential of Large Language Model Embeddings (LLMEs) with representation learning and embedding capabilities, demonstrating their promise for advancing FCRE systems.