Preserving safety alignment during large language models fine-tuning is critical, however, recent studies have demonstrated that even benign fine-tuning data may contain safety-degrading samples that silently undermine such alignment. Existing approaches typically identify safety-degrading samples using representations from a single safety-sensitive layer. However, whether a single safety-sensitive layer is sufficient for identifying such samples in multilingual settings remains unclear, where representation patterns may vary across languages. Our cross-lingual analysis shows that sensitive layers are only partially shared across languages and that safety-relevant signals often span multiple layers. Motivated by these observations, we propose MMSafe, a multi-layer framework for multilingual safety-degrading data identification that jointly leverages shared and language-specific sensitive layers. Extensive experiments across multiple models, languages, and safety benchmarks demonstrate that MMSafe reduces the average harmful-response ratio by 60% compared with random filtering and outperforms the strongest single-layer baseline, demonstrating the effectiveness of multi-layer modeling for robust multilingual safety alignment. Code is available at https://anonymous.4open.science/r/MMSAFE.

Due to the success of large-scale visual-language pretraining (VLP) models and the widespread use of image-text retrieval in industry areas, it is now critically necessary to reduce the model size and streamline their mobile-device deployment. Single- and dual-stream model structures are commonly used in image-text retrieval with the goal of closing the semantic gap between textual and visual modalities. While single-stream models use deep feature fusion to achieve more accurate cross-model alignment, dual-stream models are better at offline indexing and fast inference. We propose a Multi-teacher Cross-modality Alignment Distillation (MCAD) technique to integrate the advantages of single- and dual-stream models. By incorporating the fused single-stream features into the image and text features of the dual-stream model, we formulate new modified teacher similarity distributions and features. Then, we conduct both distribution and feature distillation to boost the capability of the student dual-stream model, achieving high retrieval performance without increasing inference complexity. Extensive experiments demonstrate the remarkable performance and high efficiency of MCAD on image-text retrieval tasks. Furthermore, we implement a lightweight CLIP model on Snapdragon/Dimensity chips with only ~100M running memory and ~8.0ms search latency, achieving the mobile-device application of VLP models.