The BabyLM shared task asks whether language models can acquire robust linguistic behavior from developmentally sized corpora. We study a causal transformer recipe for this setting that increases computation per token without adding a full stack of independent layers. Our RecGPT models reuse one transformer block recurrently, factorize the vocabulary maps to keep small models from being dominated by embeddings, and add a training-only next-latent auxiliary loss that encourages hidden states to support local transitions. We submit RecGPT-10M for Strict-Small and RecGPT-100M for Strict. Across the BabyLM 2026 zero-shot and finetuned evaluations, the submitted models improve over GPT-2 baselines and are competitive with GPT-BERT.