Dat Dao
Author directory2026
A Survey on Knowledge Distillation of Vision-Language Models
Hoang Tran Vuong | Dat Dao | Pham Khanh Chi | Minh-Phuc Truong | Quyen Tran | Linh Ngo Van | Thien Huu Nguyen | Thanh-Toan Do | Trung Le
Findings of the Association for Computational Linguistics: EMNLP 2026
Hoang Tran Vuong | Dat Dao | Pham Khanh Chi | Minh-Phuc Truong | Quyen Tran | Linh Ngo Van | Thien Huu Nguyen | Thanh-Toan Do | Trung Le
Findings of the Association for Computational Linguistics: EMNLP 2026
Vision-language models (VLMs) achieve strong multimodal performance but remain expensive to deploy. Knowledge distillation (KD) is an effective approach for compressing these models and transferring their capabilities to more efficient student models. While KD for large language models (LLMs) is already well surveyed, no existing work provides systematic coverage of KD for VLMs, where the addition of a visual modality introduces challenges that text-only distillation does not face. This paper presents the first dedicated survey of knowledge distillation for vision-language models. We propose a unified taxonomy that organizes methods along three orthogonal views: knowledge type, teacher accessibility, and structural compatibility, and use it to organize methods across major VLM distillation settings. We also survey the associated datasets and evaluation protocols, identify open challenges and outline directions for future research. We further maintain a GitHub repository at https://github.com/HoangTran223/Awesome-Knowledge-Distillation-of-VLMs to compile all surveyed papers, which we will actively update.