BabyLM is a pressure test for language-modeling assumptions: which ideas still work when the data budget is closer to a child’s linguistic input than to web-scale pretraining? We use the 2026 English tracks to ask this question in a controlled way, holding the corpus, tokenizer, backbone size family, training schedule, and public evaluation wrapper fixed while varying two token mixers and six objectives. Across 432 completed scale-grid runs on the 10M-word Strict-Small and 100M-word Strict tracks, the strongest result is also the most conservative one. Autoregressive Transformers are the best public-evaluation models at every tested scale, reaching peak fine-tuning macro scores of 64.88 and 66.94 and peak BLiMP scores of 64.70 and 75.13. Masked language modeling is the strongest non-autoregressive baseline. The negative results are informative: Gated DeltaNet improves Entity Tracking but loses grammar and broad transfer, while diffusion and flow objectives show local advantages in probes that rarely survive the official scoring interface. The lesson for BabyLM is not that newer objectives or recurrent mixers are unhelpful, but that small-data gains depend on matching the objective, architecture, sampler, and scorer.

In the small-data regime of the BabyLM Challenge, a model’s architecture still shapes what it learns. We ask how much an inductive bias toward recurrence helps, and whether current benchmarks can detect that help. Our model adapts the Hierarchical Reasoning Model, a small recurrent puzzle-solver that reuses one shared block many times per token instead of stacking more layers. We make three changes to turn it into a language model: a sequence-mixing layer that can carry order-dependent state, a gate on one of the two connections between its modules, and a tunable number of recurrent cycles. The result matches a parameter-matched GPT-2, and we use it to study three questions. Q1: recurrence buys depth without adding parameters. On a controlled state-tracking task, running more cycles of the same weights tracks longer sequences, even when the extra cycles are added only at inference. Q2: the advantage is largest where data is scarcest. leads GPT-2 on BLiMP at words, ties at , and, trained to convergence, still leads at tokens (out of budget, where data and tokenizer also change). Q3: the benchmark meant to reward state-tracking cannot see it. A null model that never reads the puzzle outscores every trained model on entity_tracking, because 28.5% of its items reveal the answer in the option list and every trained model is at chance on the rest. Gains on that task measure how well a model has learned the frequency of one answer. Because the suite misses a capability we can demonstrate directly, we recommend pairing benchmarks with controlled architectural probes.