Zero-shot text-to-speech (TTS) aims to synthesize high-quality speech for unseen speakers using only a short reference utterance. Existing non-autoregressive TTS models typically convert text tokens into speech-length representations through filler-token padding, explicit duration prediction, or text upsampling. However, these approaches either introduce redundant computation or rely on explicit alignment assumptions. We propose M3-TTS, a native two-stream multi-modal diffusion Transformer that preserves the original lengths of text and speech representations and enables direct text–speech interaction through packed joint self-attention. M3-TTS employs a dual-stage architecture consisting of Joint-DiT layers for learning text–speech correspondence and audio-only DiT layers for acoustic refinement. Reference acoustic information is injected through AdaLN-style conditioning. Experiments on the Seed-TTS benchmark demonstrate strong intelligibility and competitive speaker similarity and perceptual quality. The Mel-VAE variant further improves inference efficiency while maintaining high synthesis quality. Code and demos will be available at https://github.com/WWWWxp/M3-TTS.