Kurt MacDonald

Author directory

2026

A popular way to improve Retrieval-Augmented Generation (RAG) is to rewrite the user’s question into several variants and search with all of them. We test whether this actually helps once the underlying search is already strong. Under one fixed, competitive pipeline (BGE dense retrieval, cross-encoder reranking, and MMR diversification), we compare four query-rewriting strategies (S1–S4) against two strong LLM baselines (HyDE, Query2Doc) on three datasets (HotpotQA, AmbigNQ, and the 512K-document EnterpriseRAG-Bench) over three seeds with paired-bootstrap significance tests.

Our headline result is that rewriting alone is at best competitive with a strong baseline, but combining methods yields outsized gains because different strategies fail on different questions. A post-hoc union of four methods (S1+S3+S4+HyDE; seed-42 analysis) improves HIT@10 over the dense baseline by +12.5 points on enterprise-style data (51.70 vs 39.22), and a five-method union reaches 52.98 (+13.8 over baseline). Budget-matched controls (TopK-100 reranking, multi-pass HNSW) capture only ∼40% of this gain, confirming that complementarity, not retrieval budget, is the primary driver. On HotpotQA the union adds +1.6 to +1.8 points (p < 0.001), saturating the all-method oracle; on AmbigNQ the same fusion hurts (-2.4 below the best solo, p < 0.001), and we analyze when and why. Because rewriting is expensive (one LLM call each), we turn this into a cost-aware heuristic evaluated in simulation: a confidence-gated router that runs rewriting only when the baseline’s own top-1 score is low. It captures about half of the enterprise full-merge gain (+4.3 HIT@10 over the baseline) while paying the rewriting cost on <40% of queries, and automatically declines to rewrite on AmbigNQ, sidestepping the loss that blind merging causes. A downstream answer-quality evaluation confirms that the router improves F1 by +1.92 (p < 0.01) at roughly 40% of the expansion cost. In short: treat query rewriting as a complementary coverage source, applied through cost-aware routing, not as a standalone replacement for a strong baseline.
Every document QA system begins with a choice that is rarely studied on its own: whether to feed the model page images, extracted text, or both. We isolate this choice, holding the prompt, judge, and scoring pipeline fixed, across four commercial model endpoints, two corpora, and two context regimes (gold evidence pages and the full document). On documents that fit the image budget, page images lead on accuracy at every document length on both corpora, but this advantage carries a growing latency and cost premium: text latency stays roughly flat as documents lengthen while image latency rises steadily. Text and images also fail on different questions, with exactly one representation correct on 19-25% of items across the reported cells, so neither subsumes the other. Exploiting this complementarity, a lightweight TF-IDF router that reads only the question text gains 2.6 points over always-text while cutting median latency 30% relative to always-vision, on a document-disjoint held-out split.