Multi-agent debate among large language models (LLMs) is widely assumed to improve reasoning by letting models correct one another. We audit that assumption for English medical question answering (QA). Five open-weight 4 to 8B medical and generalist LLMs answer 8,402 questions across six benchmarks as a council, measured against the same five prompted individually and a single larger Qwen3-VL-32B, text-only. The council beats every member on five of six datasets, but most of that accuracy is in place before debate runs: 67% of verdicts settle at Round 0, and on five of six those are the more accurate questions. Where debate runs, opinion change concentrates on one anchor; on the hardest benchmark 85% of moves toward it happen while it is wrong. There, agreement is worth less than disagreement: questions all five agree on are correct 6.6% of the time against 14.6% otherwise, tracking a shared bias toward the first option. The council also trails the larger model on four of six by up to 16.2 points, at seven times the model calls for 1.6 times the compute. Agreement among models is a measurement that can be biased, not evidence of correctness.

We introduce MedBench, a benchmark for evaluating medical language models as deliberating agents rather than isolated predictors. MedBench evaluates eight models (4B?32B) on 19,625 questions from six medical QA datasets using Consensus-Aware Model Panel (CAMP), a two-tier protocol in which five 4B?8B models answer independently, revise after observing peer reasoning, and escalate persistent disagreements to larger 20B?32B models. Compared with zero-shot, few-shot, and chain-of-thought baselines, CAMP shows that deliberation is not uniformly accuracy-improving, but reveals interaction-driven behaviors hidden by single-model evaluation. On PubMedQA without external context, the 4B?8B panel outperforms the evaluated 20B?32B individual zero-shot models (54.1% vs. 33.9%), and achieves the best evaluated result with context (75.7%), suggesting that structured interaction can sometimes complement scale. Across five datasets, initial inter-model agreement is positively associated with correctness and serves as a useful difficulty signal. However, on MedXpertQA, unanimous agreement yields only 6.6% accuracy despite 14.4% overall accuracy, suggesting correlated ignorance, where shared biases make consensus misleading. Error analysis shows that most failures are debate-insufficient cases, where incorrect majorities persist despite interaction (93?97%), while debate-harmful cases account for 3?7%. MedBench positions deliberative evaluation as a complement to accuracy-centric benchmarking, measuring when model interaction corrects errors, reinforces shared mistakes, or signals the need for stronger evidence and human review.