Task-Oriented Dialogue (TOD) systems are commonly used to assist users in achieving specific goals through human-computer interactions. Existing methods typically employ a single model during response generation to simultaneously learn response style and perform fine-grained knowledge retrieval. However, these task-coupling methods often result in suboptimal responses generation and the inclusion of irrelevant knowledge in responses, making the optimization of TOD systems more difficult. To address this challenge, we propose a novel small-to-large model collaboration framework for task-oriented dialogue, named SMARTS. This framework utilizes a small language model to generate style-only responses (without knowledge), while a large language model retrieves top-1 relevant knowledge from the knowledge base independently. The style-only responses are finally filled with top-1 relevant knowledge and form the completed responses to users. Finally, a re-thinker is designed to check the language fluency and knowledge accuracy of responses. Experiments on two public datasets demonstrate that SMARTS outperforms existing methods by an average of 5.79% in Entity F1.