Evidence-Based Medical Question Answering (EB-MQA) aims to provide accurate and verifiable responses to support clinical decisions. To retrieve evidence from general medical knowledge graphs (KGs), existing knowledge augmentation methods use semantic similarity or dynamic sampling for entity matching and agentic expansion or subgraph scoring for path retrieval. These approaches can introduce entity-level noise from irrelevant or generic entities and path-level noise from redundant or spurious reasoning chains. We propose ToToFilter, a train-free topic- and topology-aware noise filtering framework with Topic-Aware Entity Localization for query-relevant entity anchoring and Topology-Aware Path Filtering for path selection based on semantic consistency, structural compactness, and information specificity. Across four MQA benchmarks, ToToFilter achieves average accuracy gains of 2.30 and 1.22 percentage points over DALK on Llama3.1-8B and Qwen2.5-7B, with maximum gains of 2.60 and 2.78 points. On PrimeKGQA, it improves answer recall and triple recall by up to 8.81 and 19.70 percentage points. Further analyses verify the effectiveness of the proposed two-level filtering framework.