We introduce HypoExplore, an agentic framework that formulates model discovery as a hypothesis-driven scientific inquiry. Given a human-specified high-level research direction, HypoExplore ideates, implements, evaluates, and improves neural architectures through evolutionary branching. Hypotheses are generated using a large language model by selecting a parent hypothesis to build upon, guided by a dual strategy that balances exploiting validated principles with resolving uncertain ones. Our proposed framework maintains a Trajectory Tree that records the lineage of all proposed architectures, and a Hypothesis Memory Bank that actively tracks associated confidence scores acquired through experimental evidence. After each experiment, multiple feedback agents analyze the results from different perspectives and consolidate their findings into hypothesis confidence updates. Under minimal research directions, HypoExplore discovers a model that achieves top results on two medical imaging tasks on MedMNIST, an improved encoder for named entity recognition on FiNER-ORD, and a lightweight 0.7M parameter model on CIFAR-10 that reaches 94.1% accuracy and transfers well to other benchmarks. Our analysis of hypothesis confidence scores across evolutionary lineages suggest that HypoExplore not only discovers stronger architectures, but can help build a genuine understanding of the design space.