Embodied agents that route every high-level decision through a single language model typically ingest the entire trajectory at every step, a regime in which prompt context grows monotonically and both lost-in-the-middle degradation and trajectory-level hallucination peak. We show that multi-agent decoupling on a shared Voronoi-Frontier blackboard is a structural answer. SkillNav exposes a sparse topology over the partially-built map whose nodes carry visual evidence and two multiplicative gates (mu_safe, mu_mem) written by specialised Safe and Memory agents; a strategic Exploration agent reads only a compact, system-prompt-style summary of the blackboard – whose size is bounded by the topology rather than by trajectory length – and writes back only a phase decision plus an SR/IG weight pair. The three agents never see one another’s prompts; false-positive evidence is suppressed architecturally by mu_mem rather than by asking the language model to revise its own past output. On HM3D val, SkillNav reaches 71.9 SR / 34.0 SPL with only 22.5 LLM calls per episode (~11.9K total tokens), surpassing the strongest published zero-shot baseline (RATE-Nav) by +4.1 SR / +2.7 SPL.