Data selection algorithms for large language model (LLM) pretraining are designed to mine ‘quality’ from the web, but that notion is murky at best. This study explores whether sociolinguistic mobility – realized through language use, in context – offers a lens for understanding what such algorithms actually select. As a case study, the FineWeb pretraining pipeline is examined for how it reshapes two tangible axes of language use: the geographical provenance and register composition of texts drawn from ICE and GloWbE.

FineWeb rejection rates are highest, and fairly uniform across provenance groups, for web-native and high-context registers (90.9% and 99.2%, respectively) but within medium-context registers, association with the United States is moderately protective: the 7% rejection rate is well below the medium-context average of 38%, an effect corroborated by NPMI. Unlike prior work on learned, classifier-based filters and US-internal sociolect bias, this study finds comparable effects in a deterministic pipeline within the global spread of English. As pretraining data selection becomes increasingly opaque, these results suggest that algorithmic ‘quality’ does not represent a real-world range of English use manifested in the web-as-corpus – a gap made visible by tangible sociolinguistic dimensions.

Entrainment, the responsive communication between interacting individuals, is a crucial process in building a strong relationship between a mental health therapist and their client, leading to positive therapeutic outcomes. However, so far entrainment has not been investigated as a measure of efficacy of large language models (LLMs) delivering mental health therapy. In this work, we evaluate the linguistic entrainment of an LLM (ChatGPT 3.5-turbo) in a mental health dialog setting. We first validate computational measures of linguistic entrainment with two measures of the quality of client self-disclosures: intimacy and engagement (p < 0.05). We then compare the linguistic entrainment of the LLM to trained therapists and non-expert online peer supporters in a cognitive behavioral therapy (CBT) setting. We show that the LLM is outperformed by humans with respect to linguistic entrainment (p < 0.001). These results support the need to be cautious in using LLMs out-of-the-box for mental health applications.