The task of Error Prediction, namely predicting whether a model output is correct, is commonly tackled with Uncertainty Quantification (UQ). However, while uncertainty metrics capture when models lack knowledge or capacity to make a prediction, they also reflect aleatoric uncertainty, inherent in the input and context. We improve error prediction for Large Language Models (LLMs) by disentangling input ambiguity from the UQ signal. We experiment on Question Answering (QA) with six UQ metrics and show that UQ metrics are more predictive of errors on unambiguous instances than on questions with multiple plausible answers. We use Gated Experts and Selective Prediction to incorporate gold and predicted ambiguity labels into the error prediction pipeline. We find that ambiguity information improves error prediction across model families, training and evaluation paradigms, and datasets (including allegedly unambiguous ones), with the gains concentrated on the UQ metrics that most entangle aleatoric and epistemic uncertainty, gaining over 10 PRR points for individual metrics.