Proceedings of the Second BabyLM Workshop

Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor 2026-10 Association for Computational Linguistics Budapest, Hungary text conference publication 979-8-89176-475-0 babylm-2026-main https://aclanthology.org/2026.babylm-main.0/ Halved CLM Exposure Mitigates Late-Training Degradation in Small Recurrent Language Models Yash Maheshwari author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 maheshwari-2026-halved https://aclanthology.org/2026.babylm-main.1/ 2026-10 1 14 Sentence-Level Foresight in Small Language Models: A Diagnostic Study of Learning Signals and Calibration Ahmet Rıfat Öztürk author Yağız Ekrem Dalar author Nedim Mutlu Sezer author Feyzi Arda Salihoğlu author Ömer Faruk Aksoy author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 ozturk-etal-2026-sentence https://aclanthology.org/2026.babylm-main.2/ 2026-10 15 23 Hard-Wiring the Induction Head: The Surprising Effectiveness of Copy Mechanisms in Data-Efficient Language Modelling Josh Taylor author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 taylor-2026-hard https://aclanthology.org/2026.babylm-main.3/ 2026-10 24 32 HydraLLM: Adaptive Width Slicing for Elastic Text and Vision-Language Models Oscar Kemme author Janek Haberer author Ali Hojjat author Olaf Landsiedel author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 kemme-etal-2026-hydrallm https://aclanthology.org/2026.babylm-main.4/ 2026-10 33 49 Can Child Language Acquisition be Modeled by Linear Causal or Bidirectional Masked Models Trained on Child-Directed Input?: A Comparison of Baby Language Models and Children in Acquiring Wanna Contraction Haerim Hwang author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 hwang-2026-child https://aclanthology.org/2026.babylm-main.5/ 2026-10 50 64 Does Typological Distance Affect Cross-lingual Transfer? Pengfei Ren author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 ren-2026-typological https://aclanthology.org/2026.babylm-main.6/ 2026-10 65 74 What Survives the BabyLM Budget? A Controlled Study of Token Mixers and Generative Objectives Filipe Laitenberger author Patrick Haller author Jonas Golde author Alan Akbik author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 laitenberger-etal-2026-survives https://aclanthology.org/2026.babylm-main.7/ 2026-10 75 92 Do Association-Reweighted BPE Merge Rules Help Data-Efficient Language Modeling? Zy Li author Phu Han author Trinh Tran author Lam Duong author Andrew Fang author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 li-etal-2026-association https://aclanthology.org/2026.babylm-main.8/ 2026-10 93 104 Retrieval Without Relevance: Prompt Dependence in Small Machine Translation Models Heesu Kim author Shota Sato author Mamoru Komachi author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 kim-etal-2026-retrieval https://aclanthology.org/2026.babylm-main.9/ 2026-10 105 122 When Does Depth Recursion Pay? A Parameter-Matched Study of Weight-Tied Hybrid Transformers on 100M Unique Words Aditya Sasidhar author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 sasidhar-2026-depth https://aclanthology.org/2026.babylm-main.10/ 2026-10 123 133 Parameter Golf: What Really Works? Prashanna Mani Paudel author Shivanand Venkanna Sheshappanavar author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 paudel-sheshappanavar-2026-parameter https://aclanthology.org/2026.babylm-main.11/ 2026-10 134 147 Prosodic Structure as a Critical Component in Phonemic Language Modelling at BabyLM Scale Bill K S Ku author Janet B Pierrehumbert author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 ku-pierrehumbert-2026-prosodic https://aclanthology.org/2026.babylm-main.12/ 2026-10 148 164 Augustinian BabyLM: What Ostensive Definition Can and Cannot Teach a Small Language Model Lisa Bylinina author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 bylinina-2026-augustinian https://aclanthology.org/2026.babylm-main.13/ 2026-10 165 178 How Far Is the Training Corpus from Real Infant Input? Auditing BabyLM Against Day-Long Home Recordings, and What an Age-of-Acquisition Curriculum Does Not Buy Imrul Huda author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 huda-2026-far https://aclanthology.org/2026.babylm-main.14/ 2026-10 179 186 Linguistically Inspired Multilingual Pre-Pretraining Masato Mita author Max Müller-Eberstein author Misaki Yamaguchi author Yohei Oseki author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 mita-etal-2026-linguistically https://aclanthology.org/2026.babylm-main.15/ 2026-10 187 198 BPE vs. Morphology-Aware Tokenization: A Case Study on Arabic Language Modeling at BabyLM Scale Mohamed Badawy author Marion Di Marco author Alexander Fraser author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 badawy-etal-2026-bpe https://aclanthology.org/2026.babylm-main.16/ 2026-10 199 208 Structural priors for data-efficient language learning Yana Veitsman author Jonas Mayer Martins author Jonathan Lautenschlager author Lisa Beinborn author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 veitsman-etal-2026-structural https://aclanthology.org/2026.babylm-main.17/ 2026-10 209 226 Arabic BabyLM: Effects of Diglossia and Tokenisation Mozafar Haider author Burcu Can author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 haider-can-2026-arabic https://aclanthology.org/2026.babylm-main.18/ 2026-10 227 239 Repetition Without Exclusivity: Scale Sensitivity of Referential Mechanisms in Child-Scale Language Models Jon-Paul Cacioli author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 cacioli-2026-repetition https://aclanthology.org/2026.babylm-main.19/ 2026-10 240 249 Reading Acquisition Order out of BabyLM Representations: Child-Scale Training Produces Developmentally Aligned Semantic Spaces Chenyi Ding author Zachary Nicholas Houghton author Masoud Jasbi author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 ding-etal-2026-reading https://aclanthology.org/2026.babylm-main.20/ 2026-10 250 259 Relational Attention for Data-Efficient Language Modeling Adrian Brasoveanu author Ece Takmaz author Jakub Dotlacil author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 brasoveanu-etal-2026-relational https://aclanthology.org/2026.babylm-main.21/ 2026-10 260 278 Can Translation Bridge the Data Gap for Low-Resource Languages? A Case Study of Hindi and Telugu Venkata Naga Sai Vishnu Rohit Pulipaka author Libby Barak author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 pulipaka-barak-2026-translation https://aclanthology.org/2026.babylm-main.22/ 2026-10 279 288 Causal, Hybrid, and Ternary Transformers under a Restricted Data Budget Arnav Srivastav author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 srivastav-2026-causal https://aclanthology.org/2026.babylm-main.23/ 2026-10 289 298 Can Randomly Permuting the LM Decoding Order Boost Data-Efficiency? Max Ploner author Alan Akbik author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 ploner-akbik-2026-randomly https://aclanthology.org/2026.babylm-main.24/ 2026-10 299 308 When the Weakest Language Keeps Moving: ARMOR-DP for Multilingual BabyLM Hanyu Liao author Yijing Chen author Tao Ji author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 liao-etal-2026-weakest https://aclanthology.org/2026.babylm-main.25/ 2026-10 309 319 Cross-Linguistic Effects in Bilingual Phoneme BabyLMs Nikitas Theodoropoulos author Maria Lymperaiou author Giorgos Filandrianos author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 theodoropoulos-etal-2026-cross https://aclanthology.org/2026.babylm-main.26/ 2026-10 320 332 TinyLibrary: Do Age-Graded Curricula Help Small Vision-Language Models? Dheeraj Varghese author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 varghese-2026-tinylibrary https://aclanthology.org/2026.babylm-main.27/ 2026-10 333 347 Not All Irregularity Is Equal: Causally Isolating a Rare Failure Mode in Japanese Morphological Inflection Wen Zhang author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 zhang-2026-irregularity https://aclanthology.org/2026.babylm-main.28/ 2026-10 348 357 TRLM: Shared-Weight Recursive Reasoning for Non-Autoregressive Language Generation Doğa Oytaç author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 oytac-2026-trlm https://aclanthology.org/2026.babylm-main.29/ 2026-10 358 368 Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM Adam Zachary Wasserman author David Beauchemin author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 wasserman-beauchemin-2026-right https://aclanthology.org/2026.babylm-main.30/ 2026-10 369 387 What Helped with Huge Data Hurt with Small Data: Standardization, Representational Geometry, and Data Scale Catherine Finegan-Dollak author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 finegan-dollak-2026-helped https://aclanthology.org/2026.babylm-main.31/ 2026-10 388 402 Compact Multilingual Pre-training under the BabyLM 100M-Word Budget with Regex-Guided Tokenization and Attention Residuals Yihan Liu author Tongjing Zhang author Pierre Nugues author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 liu-etal-2026-compact https://aclanthology.org/2026.babylm-main.32/ 2026-10 403 412 Different Corpora, Different Skills? A Paired Study of Small Language Models Edoardo Vaira author Georgios Chavales author Pierre Nugues author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 vaira-etal-2026-different https://aclanthology.org/2026.babylm-main.33/ 2026-10 413 425 Modeling the Developmental Shift in Telicity Acquisition Ellie Xia author Parisa Kordjamshidi author Alan Hezao Ke author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 xia-etal-2026-modeling https://aclanthology.org/2026.babylm-main.34/ 2026-10 426 437 Measuring Form and Function in Language Models Hector Javier Vazquez Martinez author Charles Yang author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 vazquez-martinez-yang-2026-measuring https://aclanthology.org/2026.babylm-main.35/ 2026-10 438 458 Stories for the Win: Creating and Evaluating a Synthetic Dutch Children’s Storytelling Dataset Sabijn Perdijk author Gijs Wijnholds author Suzan Verberne author Max J van Duijn author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 perdijk-etal-2026-stories https://aclanthology.org/2026.babylm-main.36/ 2026-10 459 478 Findings of the Fourth BabyLM Challenge & Workshop: Going Beyond English Leshem Choshen author Ryan Cotterell author Mustafa Omer Gul author Jaap Jumelet author Tal Linzen author Aaron Mueller author Suchir Salhan author Raj Sanjay Shah author Alex Warstadt author Ethan Gotlieb Wilcox author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 choshen-etal-2026-findings https://aclanthology.org/2026.babylm-main.37/ 2026-10 479 492 Exploring Auxiliary Objectives through Multi-Task Learning for Small-Data Language Model Pretraining: A BabyLM 2026 Submission Alonso Palomino author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 palomino-2026-exploring https://aclanthology.org/2026.babylm-main.38/ 2026-10 493 503 MoEP+: Enhancing Modular Expert Paths for Data-Efficient Language Modeling Noor Eidirhn Mustafa author Valdemar Melkersson author Pierre Nugues author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 mustafa-etal-2026-moep https://aclanthology.org/2026.babylm-main.39/ 2026-10 504 513 Four Kilobytes of Memory: A 10M-Parameter Fixed-State Recurrent LM for BabyLM Strict-Small Sam Furr author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 furr-2026-four https://aclanthology.org/2026.babylm-main.40/ 2026-10 514 524 Weight-Tied Looped Depth for Data-Constrained Pretraining Sebastian Miles author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 miles-2026-weight https://aclanthology.org/2026.babylm-main.41/ 2026-10 525 538 OLMo2 with Hybrid MLM-CLM Training for the BabyLM 2026 Multilingual Challenge Zichao Wei author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 wei-2026-olmo2 https://aclanthology.org/2026.babylm-main.42/ 2026-10 539 542 Loop Before You Leap: What Recurrent Depth Can and Cannot Buy in Data-Limited Pretraining Daichi Hotta author Yohei Kobashi author Takeshi Kojima author Yusuke Iwasawa author Yutaka Matsuo author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 hotta-etal-2026-loop https://aclanthology.org/2026.babylm-main.43/ 2026-10 543 562 URCHIN: A Horizontal Spiking Language Model for Data-Constrained Pretraining Po-Han Chiang author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 chiang-2026-urchin https://aclanthology.org/2026.babylm-main.44/ 2026-10 563 574 The Impact of Preference Optimization Data Selection: Teaching Practical Life Skills to BabyLM Barbora Štěpánková author Vojtěch Lanz author Jakub Szymanik author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 stepankova-etal-2026-impact https://aclanthology.org/2026.babylm-main.45/ 2026-10 575 584 Subword Segmental BabyLMs: Learning to Tokenise for Sample-Efficient Pretraining Francois Meyer author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 meyer-2026-subword https://aclanthology.org/2026.babylm-main.46/ 2026-10 585 597 Optimizing Multilingual BabyLM: The Roles of Tokenization, Data Packing, and Curriculum Learning Fuming Yang author Kai-hao Yang author Hsuan Lien author Lukas Edman author Alexander Fraser author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 yang-etal-2026-optimizing https://aclanthology.org/2026.babylm-main.47/ 2026-10 598 610 Pairing Original Text with Meaning-Preserving Rewrites for Sample-Efficient Small-Data Language Model Pretraining Shaoxiang Wu author Alexander Fraser author Lukas Edman author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 wu-etal-2026-pairing https://aclanthology.org/2026.babylm-main.48/ 2026-10 611 623 Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text Lukas Edman author Alexander Fraser author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 edman-fraser-2026-qwenfant https://aclanthology.org/2026.babylm-main.49/ 2026-10 624 630 Does a Data2vec Auxiliary Help Small Masked Language Models? A BabyLM-2026 Strict-Small Study Martin Benedikt author Ivan Dostál author Robin Klubarski author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 benedikt-etal-2026-data2vec https://aclanthology.org/2026.babylm-main.50/ 2026-10 631 638 Recursive Transformers for BabyLM 2026 Serdar Gülbahar author Alexander Fraser author Lukas Edman author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 gulbahar-etal-2026-recursive https://aclanthology.org/2026.babylm-main.51/ 2026-10 639 645 Spend the Wasted Epochs: Preference Post-Training from the Saturation Checkpoint for Sample-Efficient Language Modeling Ujjwal Mishra author Sudipta Roy author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 mishra-roy-2026-spend https://aclanthology.org/2026.babylm-main.52/ 2026-10 646 658 Exploring Linguistic Parameters in LMs Cristiano Chesi author Achille Fusco author Guido Formichi author Sarah Rossi author Hanna B Cygan author Tommaso Sgrizzi author Asya Zanollo author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 chesi-etal-2026-exploring https://aclanthology.org/2026.babylm-main.53/ 2026-10 659 677 Scaling Compute Instead of Parameters: Exploring Iterative Transformers for Data-Efficient Language Modeling Wiktor Kamzela author Ondrej Dusek author Mateusz Lango author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 kamzela-etal-2026-scaling https://aclanthology.org/2026.babylm-main.54/ 2026-10 678 688 Learning from Correction: Data-Efficient Language Model Training Through Adaptive Teacher-Model Feedback Sadri Islam author Tagore Rao Kosireddy author Kevin Li author Dylan Gaines author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 islam-etal-2026-learning https://aclanthology.org/2026.babylm-main.55/ 2026-10 689 701 Miss Universal Grammar: Pretraining BabyLMs with Linguistic and Non-Linguistic Sources Erika Varis Doggett author Mary Kennedy author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 varis-doggett-kennedy-2026-miss https://aclanthology.org/2026.babylm-main.56/ 2026-10 702 721 From Puzzle-Solver to Language Model: Recurrent Inductive Bias, Data Scale, and Benchmark Validity in BabyLM Patrick Haller author Filipe Laitenberger author Jonas Golde author Alan Akbik author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 haller-etal-2026-puzzle https://aclanthology.org/2026.babylm-main.57/ 2026-10 722 734 Language Allocation and Targeted Data Design for a Multilingual BabyLM Naoki Mori author Seiji Kyo author Koki Goda author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 mori-etal-2026-language https://aclanthology.org/2026.babylm-main.58/ 2026-10 735 749 Feeding BabyLMs Macaroni: Code-Switching Curricula Cause Cross-Lingual Convergence Dries Rooryck author Alexander Dazhen Cai author Yonatan Belinkov author David Alvarez-Melis author Kianté Brantley author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 rooryck-etal-2026-feeding https://aclanthology.org/2026.babylm-main.59/ 2026-10 750 766 XpertGPT: Multi-Scale Sparse Expert Routing for Data-Constrained Language Modeling Soham Jain author Harsh Singh author Divija Dewan author Atul Dev author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 jain-etal-2026-xpertgpt https://aclanthology.org/2026.babylm-main.60/ 2026-10 767 778 BabyConceptLM: Exploring Dynamic Concept Computation under Restricted Language Exposure Yizhe Xu author Ming Song author Danni He author Jianghao Liu author Yitong Wang author Qing Cai author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 xu-etal-2026-babyconceptlm https://aclanthology.org/2026.babylm-main.61/ 2026-10 779 804 Introducing Code-Switched Contexts to Cognitively-Inspired Bilingual Model Training Zhuojing Huang author Luise Pohlmann author Lisa Beinborn author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 huang-etal-2026-introducing https://aclanthology.org/2026.babylm-main.62/ 2026-10 805 822 BabySteps at BabyLM 2026: Stronger Optimization and Causal Baselines for Data-Constrained Pretraining Siddhi Satheesh author Jorge C Murillo author Yubei Chen author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 satheesh-etal-2026-babysteps https://aclanthology.org/2026.babylm-main.63/ 2026-10 823 830 Learning What Can Combine and What Can Substitute: Relational Inductive Biases for BabyLM Pretraining Yuwei Yang author Shihao Pu author Gefei Feng author Yiming Yang author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 yang-etal-2026-learning-combine https://aclanthology.org/2026.babylm-main.64/ 2026-10 831 842 Domain-Based Routing in Mixture-of-Experts Models for BabyLMs Jeremias Bohn author Sebastian Feiertag author Nicolas Raube author Emil Schwenger author Georg Groh author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 bohn-etal-2026-domain https://aclanthology.org/2026.babylm-main.65/ 2026-10 843 858 Dense Supervision, Gentle Decay: Curriculum-Guided Rebalancing of ELECTRA-Style RTD and MLM for Small-Data Pretraining Aftab Hossain Mikat author Tamjid Hasan Fahim author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 mikat-fahim-2026-dense https://aclanthology.org/2026.babylm-main.66/ 2026-10 859 874 Tokens of Meaning: Semantically-Informed Morphological Tokenization in Low-Resource Language Modeling Juan Diego Rodriguez author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 rodriguez-2026-tokens https://aclanthology.org/2026.babylm-main.67/ 2026-10 875 891 What Does Input-Dependent Depth Routing Buy in Data-Limited Language Modeling? Yupei Wang author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 wang-2026-input https://aclanthology.org/2026.babylm-main.68/ 2026-10 892 903 Language Exposure Matters: Distribution and Order in Multilingual BabyLM Pre-training JiWoo Hwang author Lukas Edman author Alexander Fraser author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 hwang-etal-2026-language https://aclanthology.org/2026.babylm-main.69/ 2026-10 904 911 Looped GPT-BERT: Trading Parameters for Computation in Small Language Modeling Tingshuo Fan author Hongtao Mu author Tianyu Zhou author Hansen Liu author Tao Ji author 2026-10 text Proceedings of the Second BabyLM Workshop Leshem Choshen editor Ryan Cotterell editor Mustafa Omer Gul editor Jaap Jumelet editor Tal Linzen editor Aaron Mueller editor Suchir Salhan editor Raj Sanjay Shah editor Alex Warstadt editor Ethan Gotlieb Wilcox editor Association for Computational Linguistics Budapest, Hungary conference publication 979-8-89176-475-0 fan-etal-2026-looped https://aclanthology.org/2026.babylm-main.70/ 2026-10 912 921