While retrieval-augmented generation (RAG) relies on retrieved evidence to mitigate hallucination, its reliability is constrained by retrieval quality and diagnosing where retrieval fails remains difficult. Existing benchmarks typically report a single aggregate score over heterogeneous query sets and are often dominated by queries whose relevant passages can be identified through simple lexical overlap, concealing the failure modes that matter most in practice. We introduce GutenBench, a fine-grained, zero-shot evaluation dataset built around a four-level retrieval taxonomy: lexical, semantic, explicit aggregation, and latent conceptual retrieval. It pairs queries with relevance judgments adjudicated by subject-matter experts, enabling precise and controlled comparisons of retrievers. We evaluate several off-the-shelf retrievers in an asymmetric, English-to-target cross-lingual setting across 272 Bible translations (141 languages) and find that performance is strong on lexical and semantic queries, but degrades when models must aggregate information across passages or recognize conceptually relevant passages with no surface overlap.