Compressing long chain-of-thought (CoT) from large language models (LLMs) is an emerging strategy to improve reasoning efficiency. Despite its promising benefits, existing studies equally compress all thoughts within a long CoT, hindering more concise and effective reasoning. To this end, we first investigate the importance of different thoughts through automatic long CoT chunking and Monte Carlo rollouts. Building on this, we propose a theoretically bounded metric to jointly measure the effectiveness and efficiency of different thoughts. We then propose Long Short, an efficient reasoning framework that enables two LLMs to collaboratively solve problems: a long-thought LLM for effectively generating important thoughts, and a short-thought LLM for efficiently generating remaining thoughts. Specifically, we synthesize cold-start data to fine-tune LLMs for long-thought and short-thought reasoning styles, respectively. Furthermore, we propose synergizing-oriented multi-turn reinforcement learning, focusing on model self-evolution and collaboration between long-thought and short-thought LLMs. Experimental results show that our method outperforms seven efficient reasoning methods and enables Qwen2.5-7B to achieve comparable performance to DeepSeek-R1-Distill-Qwen-7B, while reducing token length by over 80% across five reasoning benchmarks. Our code and data are available at https://github.com/usail-hkust/LongShort.