Instruction-tuning datasets for large language models (LLMs) are often large, redundant, and imbalanced, hindering efficient adaptation. When such data are used naively in large-batch fine-tuning, overrepresented sample groups are repeatedly included while underrepresented but informative ones remain weakly covered; in practice, large-batch training is typically implemented with data parallelism (DP) across multiple GPUs. We propose **CluSTER**, a **Clu**ster-aware balanced **S**ampling framework for **T**raining **E**fficient data **R**eduction in DP instruction tuning. We curate a representative reduced dataset through gradient-space clustering and DP-aware balanced allocation, ensuring dual-level coverage across clusters and workers, while cluster-weighted updates preserve the original data distribution. As a result, **CluSTER** reduces redundant computation and improves training stability without compromising model quality. Across multiple instruction-tuning datasets, **CluSTER** reduces training time by 64.6%, almost without accuracy loss compared to prior sampling and data reduction methods. Code is available at https://anonymous.4open.science/r/CluSTER.