Modern supervised fine-tuning (SFT) trains large language models on heterogeneous instruction mixtures, where the effective unit of optimization is an individual instruction-response pair rather than a coarse task label. Existing parameter isolation methods mitigate interference by assigning protected or trainable subspaces at the task or training stage level, but these units are often misaligned with actual parameter usage: instructions from the same task may require conflicting computations, while instructions from different tasks may share latent capabilities. We propose Instruction-Level Parameter Isolation (IPI), a fine-tuning framework that routes each instruction’s update to a sparse, capability-conditioned parameter subspace. IPI builds a multi-view instruction representation, predicts importance factors with a lightweight router, and uses an identity-trick reparameterization that preserves dense forward computation while exposing sparse base-model gradients and a task-loss saliency signal for router learning. Across heterogeneous instruction tuning benchmarks and multiple backbone models, IPI consistently improves performance over strong fine-tuning and parameter-isolation baselines. Further analyses show that instruction-conditioned masks reduce destructive gradient interactions and capture optimization structure that task labels miss.

Latent communication provides an efficient alternative to text-based exchange in multi-agent LLM systems by transmitting continuous hidden representations instead of natural-language messages. However, opacity is not privacy: latent messages may still carry private prompts, agent memory, or task-irrelevant sensitive information recoverable by downstream agents or adversaries. We study this risk through communicated latent messages, the edge-level representations that cross agent boundaries and define the privacy interface of latent multi-agent systems. We identify two coupled leakage mechanisms: instantaneous leakage from individual messages and propagation leakage accumulated across multi-round transcripts.

We propose Information-Constrained Latent Communication (ICLC), a model-agnostic communication-layer framework that constrains latent information exposure without modifying backbone LLM parameters. ICLC combines a receiver-conditioned latent bottleneck, adversarial regularization over messages and transcripts, and topology-aware gates that suppress unnecessary communication edges. Across private-context question answering, collaborative reasoning, and tool-use planning, ICLC preserves the utility benefit of latent communication while reducing average leakage from 0.747 to 0.398 compared with raw latent communication. These results suggest that privacy in latent multi-agent systems should be treated as a dynamic information-flow problem over messages, edges, and interaction rounds.