Video Large Language Models (VideoLLMs) excel in video understanding but suffer from significant computational overheads due to extensive visual tokens. Existing mainstream solutions including visual token compression and query-based Retrieval-Augmented Generation (Video RAG), face notable limitations: the former typically pursues task-agnostic compression of global video semantics, lacking query awareness, while the latter fails to optimize visual token construction and relies on costly, rigid annotation data. To address these challenges, we propose MERTA (Multi-Expert Retrieval for Token Allocation), a plug-and-play framework that integrates selective multi-expert collaborative retrieval with query-aware token allocation. MERTA selectively leverages lightweight experts to capture multimodal semantics, adaptively identifying key frames by measuring the relevance among queries, visual content, and auxiliary annotations. It then dynamically allocates the visual token budget, assigning higher budgets to key frames while keeping the total visual token budget constant. MERTA integrates seamlessly without requiring architectural modifications, demonstrating superior efficiency and performance over existing methods across multiple video understanding benchmarks.

Speech large language models (LLMs) have emerged as a prominent research focus in speech processing. In this work, we introduce VocalNet, a series of high-performance speech LLMs featuring a scalable and model-agnostic training framework as well as a novel multi-token prediction (MTP) paradigm for speech generation. We first propose an efficient two-stage training framework that enables LLMs to acquire real-time speech interaction capabilities. Through extensive experiments on various training configurations, we ensure both simplicity and effectiveness in the training strategy. Furthermore, inspired by advances in language modeling, we introduce MTP into the domain of speech LLMs—an alternative to traditional next-token prediction (NTP)—which enables the model to predict multiple future tokens at each step. Through systematic analysis and improved implementation, we show that MTP not only accelerates inference speed but also significantly enhances speech quality. Experimental results demonstrate that VocalNet achieves performance comparable to state-of-the-art Omni LLMs while outperforming existing open-source speech LLMs, despite using limited training data.