Large language models (LLMs) have demonstrated strong performance across a wide range of reasoning tasks, yet their understanding of joint and conditional probability distributions remains largely unexplored. In this work, we present a comprehensive evaluation of LLMs’ ability to reason about discrete joint and conditional probability distributions involving multiple random variables. Given observations drawn from a distribution, we evaluate models on three carefully designed tasks—mode identification, maximum likelihood estimation, and sample generation—which together capture fundamental skills required for probabilistic reasoning. Our study reveals both capabilities and limitations of current LLMs. In sample generation task, some models produce empirical distributions that closely match the target distribution despite exhibiting limited sample independence. We further observe that model performance is highly sensitive to representation: replacing explicit frequency information with raw samples leads to large accuracy drops, and even minor changes to outcome labels can further degrade performance. Together, these findings provide new insights into how LLMs reason about discrete joint and conditional probability distributions and highlight their strengths and limitations in certain probabilistic reasoning settings.

Large language models (LLMs) can now access a wide range of external tools, thanks to the Model Context Protocol (MCP). This greatly expands their abilities as various agents. However, LLMs rely entirely on the text descriptions of tools to decide which ones to use—a process that is surprisingly fragile. In this work, we expose a vulnerability in prevalent tool/function-calling protocols by investigating a series of edits to tool descriptions, some of which can drastically increase a tool’s usage from LLMs when competing with alternatives. Through controlled experiments, we show that tools with properly edited descriptions receive over 10 times more usage from GPT-4.1 and Qwen2.5-7B than tools with original descriptions. We further evaluate how various edits to tool descriptions perform when competing directly with one another and how these trends generalize or differ across a broader set of 17 different models. These phenomena, while giving developers a powerful way to promote their tools, underscore the need for a more reliable foundation for agentic LLMs to select and utilize tools and resources. Our code is publicly available at https://github.com/kazemf78/llm-unreliable-tool-preferences.