This paper studies the problem of uncertainty-aware reasoning over knowledge graphs (KGs), which aims to incorporate uncertainty quantification into unseen knowledge predictions over KGs. Previous works typically incorporate uncertainty measurement tools such as conformal predictions into graph representation learning. Despite the progress, the performance of these approaches remains limited since they usually neglect both structural information in KGs and intrinsic asymmetry between over-estimation and under-estimation. Towards this end, we propose a novel approach named Context-anchored Asymmetric Conformal Prediction for uncertainty-aware REasoning over KGs (CARE). The core of our CARE is to identify the context of each query using anchors, which guide the construction of triple-specific asymmetry prediction intervals. In particular, we first take the subset of the calibration set as a reference pool. For each triple query, we summarize the KG structural statistics into a vector, which guides the identification of its anchor samples from the pool. The confidence residuals between query and anchors are utilized to quantify over-estimation and under-estimation, producing contextual nonconformity scores. Finally, we utilize the unselected calibration dataset to determine cutoff levels for constructing asymmetric prediction intervals. Extensive experiments on benchmark datasets validate the effectiveness of CARE. The code can be found at https://github.com/fanboyang/CARE.