Multi-agent systems enhance complex reasoning but face a critical cost and effectiveness dilemma: API costs grow exponentially while performance degrades due to context degradation and error propagation. This paper proposes an evaluator-guided clause-level pruning and early halting method for lower-cost multi-agent systems named HALT, which implements a macro-to-micro intervention. At the macro-level, a difficulty aware router classifies query complexity, bypassing redundant multi-agent systems overhead for simple tasks. At the micro-level, a lightweight hindsight evaluator monitors generation in real time for complex reasoning, driving a dual-axis intervention policy: spatial pruning removes erroneous clauses, while temporal halting terminates multi-agent interaction early upon reaching a purified consensus. The extensive experiments across three different datasets demonstrate the outperformance of HALT over the baselines, achieving the 97.98%, 90.82%, 87.14% in GSM8K, MMLU, MMLU-Pro in accuracy while drastically mitigating computational overhead, with a maximum reduction of 94.3% in prompt tokens and an unprecedented reduction of 98.9% in completion tokens, which validate the effectiveness of HALT. Code is available at URL.

Instruction tuning (IT) improves large language models (LLMs) by aligning their outputs with human instructions, but its success depends critically on training data quality, and datasets such as Alpaca often contain noisy or suboptimal examples that undermine fine‐tuning. Prior selection strategies score samples using general‐purpose LLMs (e.g., GPT), leveraging their strong language understanding yet introducing inherent biases that misalign with the target model’s behavior and yield unstable downstream performance. Influence‐based methods address this by estimating each example’s marginal contribution to overall performance, but they typically assume additive contributions and therefore overlook higher‐order interactions among samples. To overcome these limitations, we propose JI²S, a novel framework that jointly models both marginal and combinatorial influences within sample groups. Applying JI²S to select the top 1,000 most influential examples from Alpaca, we fine‐tune LLaMA2‐7B, Mistral‐7B, and LLaMA2‐13B and evaluate them on Open LLM Benchmarks, MT‐Bench, and GPT‐4–judged pairwise comparisons. Our experiments show that JI²S consistently outperforms full‐dataset training and strong baselines, highlighting the value of capturing joint influence for high‐quality instruction fine‐tuning. We provide our code in this GitHub repository.