Recent studies increasingly employ large language models (LLMs) for neural architecture search (NAS); however, they typically rely on in-context learning, struggle to turn past evaluations into accumulated knowledge, and often suffer from structural hallucinations. Motivated by the observation that a search process should systematically accumulate past experience to guide future exploration, we propose CoDA-NAS, a cooperative dual-agent framework that investigates natural language as a medium for structural knowledge. At the core of our framework is a pattern knowledge base (PKB) that extracts, validates, and maintains design patterns from evaluated architectures. These patterns are retrieved as soft priors to guide the searcher agent instead of memoryless generation. The searcher agent employs a structured reasoning constraint to tightly align the generated topological outputs with these semantic concepts, thereby mitigating hallucinations. To ensure empirical reliability, an integrated lightweight predictor agent provides independent numerical validation. By applying a t-test, statistically verified knowledge enters the PKB, effectively curbing the spread of incorrect rules. Experimental results show that under strict search limits, CoDA-NAS achieves competitive performance using only a local open-weights model. Furthermore, the extracted text-based patterns generalize well to new tasks, establishing natural language as a reliable and robust medium for transferring architectural knowledge.