Yuanpei Zhao
Author directory2026
Bias-Aware Orchestration: Role-Specialized Multi-Agent Collaboration of Heterogeneous LLMs
Yuanpei Zhao | Haowei Zhang | Ziyue Qiao | Jiancheng Lv | Mao li | Wei Ju
Findings of the Association for Computational Linguistics: EMNLP 2026
Yuanpei Zhao | Haowei Zhang | Ziyue Qiao | Jiancheng Lv | Mao li | Wei Ju
Findings of the Association for Computational Linguistics: EMNLP 2026
Coordinating heterogeneous large language models (LLMs) can improve instruction following, yet current multi-agent systems often depend on manual role choices or repeated calls to one backbone. This makes it difficult to separate complementary model behavior from model bias and verbosity. We introduce Role-Specialized Multi-Agent Collaboration (RS-MA), a prompting-only framework that assigns candidate LLMs to four interpretable roles: Judge, Summarizer, Ideator, and Reasoner. RS-MA estimates role fit with audition prompts, aggregates role-specific rankings with Borda scoring, and uses the selected assignment as an inspectable role map. A refinement phase records judgment gaps and targeted edits before synthesis. On AlpacaEval 2.0, audition-selected RS-MA reaches an 88.86% length-controlled win rate, and a validated role map reaches 90.72%. A full 805-example ablation shows that single-model reuse attains high raw preference through much longer outputs, whereas the validated map improves length-controlled quality with moderate length growth. MT-Bench and FLASK further indicate preserved dialogue quality and balanced fine-grained skills. The results support explicit role assignment as a practical mechanism for combining heterogeneous LLMs when quality, completeness, and inspectable evidence are jointly required.