As large language models (LLMs) increasingly serve as the backbone of agentic systems with tool-use capabilities, predicting correctness, latency, and cost for a given tool-calling query is essential for efficient model evaluation, selection, and routing. Existing approaches predominantly adopt a semantics-first perspective, without capturing the complex interaction patterns inherent in tool-calling queries and without jointly accounting for cost and latency. We observe that query-model interaction matrices in tool-calling scenarios exhibit a low-rank structure, indicating that behavioral outcomes are governed by latent factors beyond surface semantics. We leverage this observation to propose a dual-pathway framework that decomposes interaction matrices into low-dimensional latent spaces for multi-objective prediction. The **Capability-First (CF)** pathway discovers latent structure via matrix factorization and then projects natural language queries into latent space. The **Joint Capability-Semantic (JCS)** pathway concurrently optimizes the text encoder and model embeddings, implicitly shaping query embeddings through capability-semantic interactions. Experiments on 75 LLMs demonstrate strong performance in predicting correctness, latency, and cost. Further experiments show that our method enables few-shot unseen-model generalization, parameter-efficient training, and interpretable latent representations.