Long-context inference is increasingly limited by the memory and access cost of the key-value (KV) cache. Existing KV cache compression methods face a fundamental trade-off between efficiency and information preservation. Eviction-based methods are efficient but may discard critical tokens, whereas low-rank-based methods preserve all tokens at the cost of substantial computational overhead. We propose HachiWare, a dual-branch framework that applies one-shot selective SVD to semantic-critical cache states while handling less critical states with lightweight eviction. It stores these states in a Semantic-Critical Branch (SCB) using low-rank factors, while assigning the remaining states to an Eviction Branch (EB) for cheaper eviction. During decoding, bounded active-cache refresh selectively reconstructs only active SCB entries, avoiding reconstruction of the full SCB cache. On the RULER benchmark with Llama-3.1-8B-Instruct, HachiWare incurs only a 3% accuracy drop compared with the full KV cache baseline at a KV cache compression ratio of up to 16x, while improving accuracy by 5% over existing compression methods. For decoding efficiency, HachiWare achieves up to 9.44x and 16.50x throughput speedups over Full KV at 64K and 128K context lengths. These results highlight selective allocation of reconstructability as an effective way to improve the accuracy-efficiency trade-off of KV cache compression.