Yizhe Xu
Author directory2026
BabyConceptLM: Exploring Dynamic Concept Computation under Restricted Language Exposure
Yizhe Xu | Ming Song | Danni He | Jianghao Liu | Yitong Wang | Qing Cai
Proceedings of the Second BabyLM Workshop
Yizhe Xu | Ming Song | Danni He | Jianghao Liu | Yitong Wang | Qing Cai
Proceedings of the Second BabyLM Workshop
Human language learning takes place under limited exposure and is shaped by structured regularities in the input, whereas standard Transformers apply a largely uniform sequence of computations at every token position. We ask whether dynamic latent segmentation can provide a useful inductive bias for language modeling under BabyLM-scale data constraints. BabyConceptLM causally encodes tokens, groups contextual states into learned contiguous latent segments, processes the shorter sequence with a deeper latent-level backbone, and returns to token resolution through a leakage-safe causal readout. We evaluate two profiles using recipe-matched token-only controls, with analyses of computational cost and segmentation behavior. Across English, multilingual, and Chinese comparisons, BabyConceptLM trails matched token-only controls by 0.60–1.47 points. Estimated padding-aware dense-forward FLOPs are reduced by 26% and 39% for the English and multilingual 5/8/3 models, respectively, but increase for the Chinese 2/4/2 configuration. Segmentation audits show weak alignment with word boundaries and predominantly length-driven behavior, with the larger model frequently reaching its eight-token segment cap. These results show that latent segmentation can yield a favorable compute–accuracy trade-off in some configurations, but does not by itself produce linguistically structured units or improved predictive performance. BabyConceptLM serves as a diagnostic test of hierarchical latent computation under restricted language exposure.