This paper revisits the LLM cache bandit problem, with a special focus on addressing query heterogeneity for cost-effective LLM inference. Previous works often assume uniform query sizes. Heterogeneous query sizes introduce a combinatorial structure into cache selection, making the cache replacement process more computationally and statistically challenging. We treat optimal cache selection as a knapsack problem and employ an accumulation-based strategy to effectively balance computational overhead and cache updates. In our theoretical analysis, we prove that the regret of our algorithm achieves an O(√MNT) bound, improving the coefficient of √MN compared to the O(MN√T) result in Zhu et al. (2023), where N is the total number of queries and M is the cache size. Additionally, we also provide a problem-dependent bound, which was absent in previous works. Experiments on real-world data show that our algorithm reduces the total cost by approximately 12%.