The goal of this work is to enhance balanced multimodal understanding in audio-visual large language models (AV-LLMs) by addressing modality bias without additional training. In current AV-LLMs, audio and video features are typically processed jointly in the decoder. While this strategy facilitates unified multimodal understanding, it may introduce modality bias, where the model disproportionately relies on a single dominant modality. In this work, we provide an in-depth analysis of this phenomenon and propose Fork-Merge Decoding (FMD) as a targeted solution. FMD is a simple, training-free inference strategy that rectifies this imbalance without architectural modifications. Specifically, FMD isolates audio and visual inputs in the early decoding layers (fork) to enforce modality-specific reasoning, before integrating them for joint interpretation (merge). This two-stage mechanism effectively mitigates the dominance by ensuring balanced contributions from both modalities. Experiments on VideoLLaMA2, video-SALMONN, and Qwen2.5-Omni demonstrate that FMD consistently enhances multimodal understanding across diverse benchmarks.