Multi-agent systems powered by large language models (LLMs) increasingly decompose tasks by passing natural-language relays among specialized agents. This creates a boundary-level privacy failure: a non-entitled internal agent can learn private context before any final-output guard or tool-action checker observes a violation. We introduce contextual-integrity mediation for inter-agent communication, a formulation in which each candidate relay is treated as a commit that must satisfy recipient entitlement, attribute minimization, and transmission conditions before delivery. We present CI-Relay, a runtime mediation layer that induces CI scopes from runtime-visible norms and relay metadata, packages candidate messages as CI-labeled envelopes, projects recipient-scoped views, propagates transmission constraints, and records auditable mediation traces. On relay traces converted from three CI benchmarks, unmediated relays leak internally in 52.2% of cases. Relative to a metadata-matched LLM Guard Relay that receives the same induced scope, CI-Relay reduces internal leakage from 12.4% to 1.8% and downstream leakage from 10.9% to 5.6%, while improving utility and reducing over-redaction. Gains hold across three live LLM backends; multi-hop, prompt-robustness, and ablation studies show that the gains arise from structured CI mediation rather than generic privacy prompting, guard rewriting, or final-output checking.

Self-attention mechanisms in transformers enable tokens to interact across a sequence but lack an explicit inductive bias to capture local contextual dependencies, an inherent characteristic of natural languages. We propose Token-Wise Kernels (TWiKers), a novel enhancement to transformers that learn token-specific convolutional kernels applied to the keys or values. Each token is assigned a small kernel, initialized to the “Central Dirac” (e.g., [0,1,0] for size=3), meaning the token “bears” the attention from all other tokens alone. During training, these kernels adapt, and greater deviation from the Central Dirac indicates stronger attention redistribution to neighboring tokens. This introduces the first transformer weights with direct semantic interpretability. Our experiments show that content words (e.g., nouns and verbs) retain self-focus, while function words (e.g., prepositions and conjunctions) shift attention toward their neighbors, aligning with their syntactic and semantic roles. We further apply TWiKers to distinguish literary genres, historical periods, and authors, demonstrating their effectiveness in capturing high-level stylistic patterns. Finally, we demonstrate the potential of TWiKers as an effective inductive bias to improve transformer training, validated across a range of downstream tasks.