Children learn words in a strikingly consistent order, and much of this order can be predicted from word-level properties such as frequency and concreteness. We ask whether acquisition order information is also implicitly present in the internal representations of language models, and what linguistic factors drive this. We probe 18 public checkpoints that span three architecture families at matched size, data scales from 10 million to roughly 180 billion words, and the winning systems and official baselines of the three previous BabyLM challenges. A ridge read-out of frozen hidden states predicts Wordbank age of acquisition for 619 CDI words, with the probing layer chosen on training folds only. Models trained on child-scale data match or exceed their full-scale counterparts in every family. Masked and hybrid training objectives yield more aligned spaces than purely causal training, and challenge winners are not the most aligned systems. Geometric measures of semantic structure add predictive value beyond classic predictors only for child-scale models. AoA decodability thus provides a training-free measure of developmental alignment for BabyLM evaluation.

To what extent do large language models learn abstract representations as opposed to more superficial aspects of their very large training corpora? We examine this question in the context of binomial ordering preferences involving two conjoined nouns in English. When choosing a binomial ordering (radio and television vs television and radio), humans rely on more than simply the observed frequency of each option. Humans also rely on abstract ordering preferences (e.g., preferences for short words before long words). We investigate whether large language models simply rely on the observed preference in their training data, or whether they are capable of learning the abstract ordering preferences (i.e., abstract representations) that humans rely on. Our results suggest that both smaller and larger models’ ordering preferences are driven exclusively by their experience with that item in the training data. Our study provides further insights into differences between how large language models represent and use language and how humans do it, particularly with respect to the use of abstract representations versus observed preferences.