Multi-intent spoken language understanding remains challenging due to the scarcity of labeled data and the complexity of modeling intent-slot correlations. In this paper, we propose Collaborative Self-Distillation (Colla-SD), a novel framework that leverages a pair of open-weight large language models (LLMs) to synthesize high-quality labeled samples, which are used for fine-tuning the LLMs to achieve self-distillation. Our approach consists of six stages: (a) intent-slot sampling, (b) reverse progressive skeleton generation, (c) deduplication and quality filtering, (d) cross-model consensus flagging, (e) adjudication of disagreements, and (f) model training. Evaluations on the test sets of public benchmarks show that Colla-SD dramatically improves the open-weight LLM’s overall accuracy by up to 13 times. Equipped with Colla-SD, 7B/8B open-weight models achieve up to 62.68% intent accuracy and 66.13% slot F1 on MixATIS, even outperforming massive open-source and commercial LLMs while also enabling full joint prediction. Ablation studies confirm that each component of our method is critical for the quality of synthesized data, and further analysis reveals that our method can generalize to diverse LLM architectures. Our work demonstrates that Colla-SD can effectively bridge the gap between LLM generation and zero-shot structured semantic parsing.

Knowledge base question answering (KBQA) aims to answer natural language questions by reasoning over structured knowledge bases. Existing approaches often struggle with the complexity of mapping questions to precise logical forms, particularly when dealing with diverse entities and relations. In this paper, we propose Hierarchical Topology Multi-task Learning (HTML), a novel framework that leverages a hierarchical multi-task learning paradigm to enhance the performance of logical form generation. Our framework consists of a main task: generating logical forms from questions, and three auxiliary tasks: entity prediction from the input question, relation prediction for the given entities, and logical form generation based on the given entities and relations. Through joint instruction-tuning, HTML allows mutual guidance and knowledge transfer among the hierarchical tasks, capturing the subtle dependencies between entities, relations, and logical forms. Extensive experiments on public benchmarks show that HTML markedly outperforms both supervised fine-tuning methods and training-free ones based on powerful large language models (e.g., GPT-4), demonstrating its superiority in question understanding and structural knowledge reasoning.

Relation Triplet Extraction (RTE) is a fundamental while challenge task in knowledge acquisition, which identifies and extracts all triplets from unstructured text. Despite the recent advancements, the deep integration of the entity-, relation- and triplet-specific information remains a challenge. In this paper, we propose a Graph-based Mixture-of-Experts mutual learning framework for RTE, namely RTE-GMoE, to address this limitation. As a model-agnostic framework, RTE-GMoE distinguishes itself by including and modeling the mutual interactions among three vital task-specific experts: entity expert, RTE expert, and relation expert. RTE expert corresponds to the main RTE task and can be implemented by any model and the other two correspond to the two auxiliary tasks: entity recognition and relation extraction. We construct an expert graph and achieve comprehensive and adaptive graph-based MoE interactions with a novel mutual learning mechanism. In our framework, these experts perform knowledge extractions collaboratively via dynamic information exchange and knowledge sharing. We conduct extensive experiments on four state-of-the-art backbones and evaluate them on several widely-used benchmarks. The results demonstrate that our framework brings consistent and promising improvements on all backbones and benchmarks. Component study and model analysis further verify the effectiveness and advantages of our method.