Languages differ in how much of a word’s meaning is visible in its spelling. A colour term may name its own category in its written form or give no hint of it at all, and languages differ widely in how many of their terms do. This matters for a standard test. A model is said to represent colour well if it places two colour words close together whenever the colours they name look alike, and far apart when they do not. Comparing two languages means running that test on each language’s own colour words. But where the spelling already names the category, a model can place the words correctly by reading their spelling alone, so the test is easier in some languages than in others. We quantify this for four languages and two multilingual models, Qwen2.5-7B and Llama-3.1-8B. The proportion of terms whose written form names their category ranges from 0% to 92% across our four lexicons, and the proportion of the score that disappears once written-form similarity is removed tracks it closely (r = 0.84): 0–5% in Spanish and English, 32–71% in Chinese, 37–42% in German. On the raw test Chinese scores higher than English. Counting only what the model could not have got from spelling, English is ahead. We then test all four languages on the same 43 colours, so that no vocabulary gives more away than any other. Their scores become close enough that we cannot separate them at this sample size, though a finer comparison shows the four languages still arrange those colours somewhat differently. Much of what a conventional comparison reports then looks like a fact about how each language spells its colour words rather than about the model.

Despite strong performance on standard benchmarks, it remains unclear whether large language models (LLMs) evaluate social pragmatics in ways that align with human judgments. We evaluate LLM politeness judgments using two English-language datasets with complementary annotation formats: continuous human ratings and three-way categorical labels. Across the seven evaluated models, we find that inter-model agreement is stronger than model-human agreement. Strategy-level analyses suggest that model-human alignment is associated with explicit linguistic cues, while some rapport-building strategies occur more frequently in misaligned cases. In the categorical task, model predictions exhibit systematic neutral compression, characterized by the overproduction of Neutral labels and the underprediction of Impolite labels. This pattern persists when expert consensus is used as the reference on a diagnostic subset. Our findings highlight the need for pragmatic evaluations that go beyond aggregate agreement metrics by examining directional patterns of model-human disagreement across different human references.