Heesu Kim
Author directory2026
Retrieval Without Relevance: Prompt Dependence in Small Machine Translation Models
Heesu Kim | Shota Sato | Mamoru Komachi
Proceedings of the Second BabyLM Workshop
Heesu Kim | Shota Sato | Mamoru Komachi
Proceedings of the Second BabyLM Workshop
We study how a small machine translation model should allocate a fixed data budget between parametric training and retrieval memory. We define a shared 100M-token corpus-access ledger that charges pretraining, fine-tuning, and retrieval-datastore tokens alike. Using ≈42M-parameter models, we compare three uses of the same per-pair datastore across 12 directed pairs of English, Chinese, Japanese, and Korean: leaving it unused, training on it, or retrieving from it into the prompt. We evaluate retrieval additivity across all 12 pairs and analyze retrieval-conditioned behavior in a four-pair core set. Retrieval shows no consistent additive gain over simply not using the datastore, a result that persists under an off-budget dense retriever and pools up to approximately 1000× larger. On the four-pair core set, retrieval-trained models are nevertheless highly context-dependent: removing retrieved examples causes large score drops, while replacing them with query-unrelated, count-matched same-pair examples recovers nearly the full score. This suggests that the learned dependence is primarily on exemplar-shaped context rather than on query-specific information carried by the retrieved neighbors. Our training and evaluation code is available at https://github.com/kheesu/retrieval-without-relevance.