Mixture-of-Experts (MoE) Transformers improve scaling efficiency by activating only a subset of experts per token, but deployed inference remains constrained by memory capacity and bandwidth. Existing expert compression methods face a quality–memory–latency trilemma: pruning or fully merging degrades model quality, while partial merging increases per-token activated parameters and hurt latency. We propose NimbleMoE, a training-free, collaboration-aware partial expert merging framework that reduces both model size and activated-parameter cost. NimbleMoE clusters experts by routing co-activation so co-activated experts share a low-rank subspace. To compensate for the capacity lost, NimbleMoE adds hardware-efficient Monarch residuals, with an alignment-aware choice between latent-space and output-space compensation. Across multiple MoE backbones, NimbleMoE preserves model quality under 20–25% total-parameter compression and outperforms prior pruning and merging baselines, while delivering up to 1.89× MoE-MLP kernel speedup in bandwidth-bound settings, 1.31× end-to-end TPOT speedup, and 2.81× expert-parallel MoE-kernel speedup.