Dheeraj Varghese

Author directory

2026

Developmentally plausible curricula are a recurring hypothesis in research on cognitively plausible language-model training, but it remains unclear which notions of difficulty produce useful training curricula. We ask whether annotations derived from children’s literature yield a usable ordering signal. We introduce TinyLibrary, a corpus of 46K illustrated children’s-book pages drawn from a collection for readers aged 3-13. We infer each book’s age band from its OCR text and use the resulting label to condition the generation of multimodal training data. This design lets us test whether the developmental structure of children’s literature can be used to generate synthetic multimodal training data as an effective curriculum signal. We train small vision-language models for up to 10 epochs, using a pretrained vision encoder and a language model trained from scratch, and compare five curriculum orderings at matched token budgets. We evaluate the resulting models on the zero-shot BabyLM 2026 suite. Our results show that the age progression only partly aligns with teacher-model predictability: readability increases monotonically as intended, whereas teacher perplexity is U-shaped, with text for the youngest readers among the least predictable per token. Consistent with this mismatch, age-based ordering shows no reliable advantage over random ordering. Teacher-perplexity ordering achieves the highest mean BLiMP score and exceeds its seed-matched random counterpart in 10 of 12 comparisons, but this advantage transfers unevenly across teacher families and is entangled with annotation format. No ordering yields a reliable grounding advantage: Winoground remains at chance, while VQA is above chance but variable across conditions and seeds. We release the annotations and code.
Search
Co-authors
    Venues
    Fix author