Task-Oriented Dialogue (TOD) systems are commonly used to assist users in achieving specific goals through human-computer interactions. Existing methods typically employ a single model during response generation to simultaneously learn response style and perform fine-grained knowledge retrieval. However, these task-coupling methods often result in suboptimal responses generation and the inclusion of irrelevant knowledge in responses, making the optimization of TOD systems more difficult. To address this challenge, we propose a novel small-to-large model collaboration framework for task-oriented dialogue, named SMARTS. This framework utilizes a small language model to generate style-only responses (without knowledge), while a large language model retrieves top-1 relevant knowledge from the knowledge base independently. The style-only responses are finally filled with top-1 relevant knowledge and form the completed responses to users. Finally, a re-thinker is designed to check the language fluency and knowledge accuracy of responses. Experiments on two public datasets demonstrate that SMARTS outperforms existing methods by an average of 5.79% in Entity F1.

In real-world task-oriented dialogue (TOD), users often express their intents through both text and images, with images providing essential cues for fine-grained requirements. Addressing such scenarios demands effective cross-modal semantic modeling and precise knowledge retrieval. However, existing TOD methods primarily focus on text and assume one-to-one attribute mappings, failing to reflect the multimodal nature of real-world information. To overcome these limitations, this paper proposes an end-to-end vision-guided retrieval-generation framework named INSIGHT. It introduces a progressive dynamic fusion mechanism that performs layer-wise cross-modal interaction to achieve robust multimodal dialogue understanding. In addition, a graph-structured entity modeling strategy explicitly models one-to-many image attribute mappings through distributed latent semantic learning, thereby preserving visual diversity. Finally, knowledge retrieval and response generation are jointly optimized in an end-to-end manner. To evaluate real-world scenarios, we construct VISTA, a new multimodal TOD dataset that features image-driven multi-turn dialogues and supports one-to-many entity attribute mappings. Experimental results on two multimodal datasets demonstrate that INSIGHT consistently outperforms existing methods across various backbone models.

Sight-singing is a fundamental yet challenging task in music education, where timely and fine-grained feedback is crucial for helping learners identify mistakes and improve their performance efficiently. However, existing automatic methods mainly focus on isolated subtasks such as alignment or scoring, and are unable to provide end-to-end diagnostic feedback with explicit identification of pitch, rhythm, and musical fluency errors. In this paper, we propose Cantus, a novel multimodal large language model framework for sight-singing assessment and exercise recommendation generation. Cantus jointly models vocal audio and symbolic musical representations, and leverages a pedagogically grounded error representation module together with a large language model to diagnose performance errors, generate structured assessment comments, and provide personalized exercise recommendations. To support this task, we construct a new dataset MIVO (MIDI-Voice) containing 1,003 real-world sight-singing samples with aligned MIDI notes and vocal recordings collected from authentic teaching scenarios. Experimental results demonstrate the effectiveness of Cantus in sight-singing assessment, achieving strong performance across multiple metrics. Human evaluation of feedback quality shows the potential of Cantus in supporting student-centered sight-singing learning.

Knowledge retrieval and response generation are fundamental to task-oriented dialogue systems. However, dialogue context frequently contains noisy or irrelevant information, leading to sub-optimal result in knowledge retrieval. One possible approach to retrieving knowledge is to manually annotate standard queries for each dialogue. Yet, this approach is hindered by the challenge of data scarcity, as human annotation is costly. To solve the challenge, we propose an LLM-enhanced model of query-guided knowledge retrieval for task-oriented dialogue. It generates high-quality queries for knowledge retrieval in task-oriented dialogue solely using low-resource annotated queries. To strengthen the performance correlation between response generation and knowledge retrieval, we propose a retrieval preservation mechanism by further selecting the most relevant knowledge from retrieved top-K records and explicitly incorporating these as prompts to guide a generator in response generation. Experiments on three standard benchmarks demonstrate that our model and mechanism outperform previous state-of-the-art by 3.26% on average with two widely used evaluation metrics.

Spoken-only languages are languages without a writing system. They remain excluded from modern Natural Language Processing (NLP) advancements like Large Language Models (LLMs) due to their lack of textual data. Existing NLP research focuses primarily on high-resource or written low-resource languages, leaving spoken-only languages critically underexplored. As a popular NLP paradigm, LLMs have demonstrated strong few-shot and cross-lingual generalization abilities, making them a promising solution for understanding and translating spoken-only languages. In this paper, we investigate how LLMs can translate spoken-only languages into high-resource languages by leveraging international phonetic transcription as an intermediate representation. We propose UNILANG, a unified language understanding framework that learns to translate spoken-only languages via in-context learning. Through automatic dictionary construction and knowledge retrieval, UNILANG equips LLMs with more fine-grained knowledge for improving word-level semantic alignment. To support this study, we introduce the SOLAN dataset, which consists of Bai (a spoken-only language) and its corresponding translations in a high-resource language. A series of experiments demonstrates the effectiveness of UNILANG in translating spoken-only languages, potentially contributing to the preservation of linguistic and cultural diversity. Our dataset and code will be publicly released.

Figurative language in media such as memes, art, or comics has gained dramatic interest recently. However, the challenge remains in accurately justifying and explaining whether an image caption complements or contradicts the image it accompanies. To tackle this problem, we design a modal-supplement framework MAPPER consisting of a describer and thinker. The describer based on a frozen large vision model is designed to describe an image in detail to capture entailed semantic information. The thinker based on a finetuned large multi-modal model is designed to utilize description, claim and image to make prediction and explanation. Experiment results on a publicly available benchmark dataset from FigLang2024 Task 2 show that our method ranks at top 1 in overall evaluation, the performance exceeds the second place by 28.57%. This indicates that MAPPER is highly effective in understanding, judging and explaining of the figurative language. The source code is available at https://github.com/Libv-Team/figlang2024.