Debate-to-Skill: Capability-Bound Process Supervision for Industrial Query-to-Agent Annotation

Shiyu Zhang, Leisheng Cheng, Huifu Li


Abstract
Industrial query-to-agent matching fails when topical relevance is mistaken for executable capability, especially on long-tail and boundary-sensitive requests. We formulate annotation as capability-bound process supervision and instantiate it with Debate-to-Skill, which uses reusable decision principles, structured deliberation, verifier-based verdict extraction, and disagreement-driven refinement. On an industrial Query2Agent benchmark, we compare Debate-to-Skill with direct-label supervision, reasoning-SFT, and structural ablations. The results test whether gains come from supervising the capability-critical decision process itself, especially on grey-zone cases where semantic relatedness and executable capability diverge.
Anthology ID:
2026.emnlp-industry.6
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
70–78
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.6/
DOI:
Bibkey:
Cite (ACL):
Shiyu Zhang, Leisheng Cheng, and Huifu Li. 2026. Debate-to-Skill: Capability-Bound Process Supervision for Industrial Query-to-Agent Annotation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 70–78, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Debate-to-Skill: Capability-Bound Process Supervision for Industrial Query-to-Agent Annotation (Zhang et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.6.pdf