Multi-agent LLM systems can outperform single-agent baselines, yet practitioners still cannot predict which design works for a new task or diagnose why one fails. We argue this gap persists largely because the field lacks a diagnostic framework with measurable primitives and testable predictions. We introduce DIANOIA, a three-channel decomposition of multi-agent reasoning gain into coverage, fidelity, and synthesis, each of which is empirically measurable. From this decomposition, we propose a diagnostic protocol that forms qualitative hypotheses about bottleneck channels for tasks with measurable quality signals. We instantiate the protocol as a multi-agent system whose three components mirror the channels: role-diverse proposers for coverage, execution- or pseudo-verification-grounded review for fidelity, and iterative synthesis. On GSM8K, MBPP, and BFCL-SP, our method outperforms strong multi-agent baselines under matched standard configurations; on MBPP, it matches the strongest baseline with approximately 5× fewer tokens and dominates the measured accuracy–token Pareto frontier. Full-scale MBPP replications on Qwen, DeepSeek, and Gemini consistently improve over the corresponding single-model baseline. A prospective held-out evaluation on VitaBench, whose bottleneck hypothesis was recorded before running DIANOIA, improves pass rate from 29.5% to 45.0%. DIANOIA reframes multi-agent design as channel-aware resource allocation: diagnose the likely bottleneck, then invest tokens accordingly.

Self-improvement methods enable large language models (LLMs) to generate solutions themselves and iteratively train on filtered, high-quality rationales. This process proves effective and reduces the reliance on human supervision in LLMs’ reasoning, but the performance soon plateaus. We delve into the process and find that models tend to over-sample on easy queries and under-sample on queries they have yet to master. As iterations proceed, this imbalance in sampling is exacerbated, leading to a long-tail distribution where solutions to difficult queries almost diminish. This phenomenon limits the performance gain of self-improving models. A straightforward solution is brute-force sampling to balance the distribution, which significantly raises computational costs. In this paper, we introduce Guided Self-Improvement (GSI), a strategy aimed at improving the efficiency of sampling challenging heavy-tailed data. It leverages Socratic-style guidance signals to help LLM reasoning with complex queries, reducing the exploration effort and minimizing computational overhead. Experiments on four models across diverse mathematical tasks show that GSI strikes a balance between performance and efficiency, while also being effective on held-out tasks.