Exposing latent lexical overlap, script romanization has emerged as an effective strategy for improving cross-lingual transfer (XLT) in multilingual language models (mLMs). Most prior work, however, focused on setups that favor romanization the most: (1) transfer from high-resource Latin-script to low-resource non-Latin-script languages and/or (2) between genealogically closely related languages with different scripts. It thus remains unclear whether romanization is a good representation choice for pretraining general-purpose mLMs, or, more precisely, if information loss associated with romanization harms performance for high-resource languages. We address this gap by pretraining encoder LMs from scratch on both romanized and original texts for six typologically diverse high-resource languages, investigating two potential sources of degradation: (i) loss of script-specific information and (ii) dilution of language-specific representations from increased subword overlap. Using two romanizers with different fidelity profiles, we observe negligible performance loss for languages with segmental scripts, whereas languages with morphosyllabic scripts (Chinese and Japanese) suffer degradation that higher-fidelity romanization mitigates but cannot fully recover. Importantly, comparing monolingual LMs with their mLM counterpart, we find no evidence that increased subword overlap dilutes language-specific representations. We further show that romanization improves encoding efficiency (i.e., fertility) for segmental scripts at a negligible performance cost.

Phonological competence - knowledge of how surface characters encode underlying pronunciation - underpins applications from poetry generation to speech recognition. Although text-only Large Language Models (LLMs) receive no direct phonetic supervision, prior work shows that they nonetheless develop such competence in English. It remains, however, unclear (1) whether this generalizes across languages and scripts and (2) how this text-based phonological knowledge interacts with the audio processing competencies of speech-augmented LLMs (SLLMs). In this work, we introduce mPhonBench, a massively multilingual benchmark that probes text-only and speech-augmented LLMs for phonological knowledge, spanning 10 tasks, 58 diverse languages, and 19 scripts. While mPhonBench is text-centric, we complement its nine text-only tasks with word-level phone recognition (PR) to jointly analyse text- and speech-based phonological knowledge in SLLMs. We evaluate a wide selection of open-weight LLMs on mPhonBench, and find that: (i) mid-sized LLMs match or surpass expert systems on grapheme-to-phoneme conversion (G2P) and generalize beyond memorization; (ii) underrepresented languages incur a performance penalty; (iii) while this penalty can be partially offset via in-context learning with few task examples, models struggle to exploit, in-context provided, general phonological descriptions of languages; and (iv) frontier SLLMs are competitive with expert models on word-level PR, but the capability is not linked to model size, suggesting that training regimes play a key role in developing audio-based phonological competence.