Instruction-tuning datasets for large language models (LLMs) are often large, redundant, and imbalanced, hindering efficient adaptation. When such data are used naively in large-batch fine-tuning, overrepresented sample groups are repeatedly included while underrepresented but informative ones remain weakly covered; in practice, large-batch training is typically implemented with data parallelism (DP) across multiple GPUs. We propose CluSTER, a Cluster-aware balanced Sampling framework for Training Efficient data Reduction in DP instruction tuning. We curate a representative reduced dataset through gradient-space clustering and DP-aware balanced allocation, ensuring dual-level coverage across clusters and workers, while cluster-weighted updates preserve the original data distribution. As a result, CluSTER reduces redundant computation and improves training stability without compromising model quality. Across multiple instruction-tuning datasets, CluSTER reduces training time by 64.6%, almost without accuracy loss compared to prior sampling and data reduction methods. Code is available at https://anonymous.4open.science/r/CluSTER.