With the recent commercial availability of Large Language Models (LLMs), end users increasingly turn to them for encyclopedic knowledge. Doing so, users may inadvertently inject incorrect assumptions (as presuppositions) into LLM prompts, which are then affirmed by LLMs. In this work, we investigate the effect that existential presuppositions have on LLM responses. We introduce the Dataset for Evaluating and Explaining the Effect of Existential Presuppositions (D4EP) and use it to quantify their effect. We further develop a probing-based explanation that analyzes the LLM as a classifier and finds that the effect of presuppositions corresponds to a change of the classifier’s operating point.