Large Language Models (LLMs) encode parametric knowledge but can exhibit format bias when resolving knowledge conflicts. When evidence is rendered as text, tables, or knowledge graphs, LLMs can favor certain formats despite unchanged semantic content. Existing work mainly diagnoses this bias, while existing inference-time mitigation remains tied to a single rendered prompt. We propose FaDe, a cross-format knowledge distillation framework that treats evidence format as a controllable nuisance variable and marginalizes teacher predictions over controlled format assignments. FaDe constructs a format-marginalized soft target from these renderings, then trains the student with an asymmetric two-path objective that combines heterogeneous-format debiasing with all-text anchoring. Experiments on ConflictBank and two QA-derived controlled conflict sets with four model backbones show FaDe reduces format bias while improving dual-source coverage and cross-format stability under single-pass inference. Our code is available at https://github.com/Kaia91/FaDe.