Misaki Yamaguchi

Author directory

2026

Encoding Universal Grammar principles (UG) during pre-pretraining (PPT) has been shown to produce inductive biases in language models, which facilitate natural language acquisition. While UG principles should be language-agnostic, each language follows different typological parameters. To empirically validate this theory, we operationalize three levels of typological flexibility via PPT: non-typological k-shuffle Dyck, as well as MP-STRUCT, which encodes the Minimalist Program operations MERGE, AGREE, MOVE following fixed typological parameters. Additionally, we introduce MP-STRUCT BCC (BCC), which parameterizes typological features following the Borer–Chomsky Conjecture to allow for a more typologically flexible parametrization. Through a multilingual, subtask-level evaluation across 207 grammatical tasks for English, Dutch, and Chinese, we find that principles-driven PPT significantly improves average accuracy, but selectively degrades language-specific phenomena such as wh-island constructions. In contrast, structure-only PPT, as well as typologically flexible PPT eliminate wh-island degradation while preserving overall gains, demonstrating that typological parameterization offers a path toward multilingual syntactic priors. Exploratory model-internal analyses are consistent with these findings at the levels of output probabilities, attention structure, as well as cross-lingual representation alignment.