Jon-Paul Cacioli

Author directory

2026

We show that structural grammatical priors produce targeted, linguistically specific effects on grammatical learning: improving filler-gap dependencies — which require long-distance hierarchical tracking — by 9–13 percentage points beyond structural regularisation alone (d = 2.41–2.82), while damaging locally cued phenomena regardless of whether the grammar is real or random. This phenomenon-specificity, revealed by a random grammar control, suggests the right question is not whether structural priors help, but for which constructions and why. We test this by augmenting BabyBERTa (7.4M parameters) with a differentiable PCFG auxiliary loss derived from Minimalist Grammar, trained on AO-CHILDES (893K sentences of child-directed speech). In a pre-registered study of 190 experimental runs spanning 7 constraint strengths, 3 data scales, 5 random seeds, and 3 independent lexicon permutations, our confirmatory hypotheses about overall accuracy and sample efficiency are falsified. However, a random grammar control (n = 15 runs per condition; three independent lexicon permutations) reveals that linguistically accurate category assignments specifically drive filler-gap gains: real grammar outperforms both a structurally equivalent random grammar and the no-grammar baseline, while both conditions equally damage subject-verb agreement. These results show that structural priors function as targeted interventions rather than global boosters: they help specifically the constructions, specifically long-distance dependencies, whose computational demands align with what phrase-structure representations encode. We release code and pre-registered materials.
We present the first systematic evaluation of mutual exclusivity (ME), the bias to map novel words to novel referents, in text-only language models trained on child-directed speech. We operationalise ME as referential suppression and train 45 GPT-2 models (2.9M–33.5M parameters; 5–20 epochs on AO-CHILDES; 5 seeds) on a pre-registered ME battery. Across all 9 cells of the grid, models show robust anti-ME repetition priming, the opposite of ME, in 85–100% of items (all p < 2.5 × 10⁻¹³), attenuating with improved language modelling (Spearman ρ = −0.533) but never crossing zero. A positive control on pretrained GPT-2 (124M–1.5B) shows the same priming, attenuating with scale but never reversing into significant ME, confirming the battery can detect suppression when present. The dissociation also holds on a size-matched non-CDS corpus (WikiText). A context-dependence diagnostic further shows that apparent ME-like patterns with nonce tokens reflect embedding similarity, not referential disambiguation. We connect this to the grounded cognition literature, where ME does emerge in visually grounded models, and argue that referential grounding, or another mechanism instantiating competition among candidate referents, may be required for ME-like behaviour to emerge.
Search
Co-authors
    Venues
    Fix author