Modern Large Language Models (LLMs) are often criticized for producing repetitive and homogeneous text, despite possessing vast latent vocabularies. While previous research has focused on model knowledge and training data, we investigate the role of decoding mechanics in suppressing linguistic diversity. It is well-known that sampling filters cut the tail of the token probabilities, but the specific impact at the word level remains unexplored. In this paper, we introduce the Word Coverage Score (WCS), a metric that quantifies the extent to which contextually appropriate human vocabulary is mathematically pruned by standard sampling filters (e.g., Top-p, Top-k, and Min-p). Rather than assessing static knowledge, the WCS measures the lexical survival rate of words as a function of sampling parameters through a deterministic Forced-Path Audit. By auditing 13 open-weight models on 10,000 evaluation samples per model from the modern FineWeb corpus at four temperatures, we identify which human lexical choices are rendered unreachable by the decoder, even when they reside within the underlying probability space. To characterize word usage reduction without relying only on absolute erasure, we additionally introduce the Low-Coverage Word Rate (LCWR), the fraction of target words reachable in fewer than a specified proportion of contexts. In matched downstream generation experiments, greater WCS is associated with greater output-text lexical diversity: repeated-measures Spearman correlations are 𝜌=.67 for Type-Token Ratio (TTR), 95% Confidence Interval [.50, .82] and 𝜌 = .66 for Measure of Textual Lexical Diversity (MTLD), 95% Confidence Interval [.51, .81]. These results provide quantitative evidence regarding how inference configurations shape vocabulary compression, establishing WCS as a computationally efficient diagnostic tool for hyperparameter debugging and vocabulary calibration.

Current multilingual benchmarks cover only a fraction of the languages represented in modern pretraining corpora and often conflate language proficiency with the ability to answer questions grounded in specific regions and communities. We introduce INCLUDE-2.0, a large-scale benchmark for region-aware multilingual factual evaluation, which covers 91 languages from 86 countries (127 total language–country pairs), comprising over 220K curated samples drawn from native regional sources. Unlike existing benchmarks that often treat language as a proxy for region, IINCLUDE-2.0 distinguishes between questions whose answers are independent of context, vary by country, concern widely documented facts about a place, or depend on knowledge primarily expressed through local linguistic and cultural sources. Each item is annotated with its regional knowledge type and domain, and paired with an English translation, enabling controlled analysis of the relationship between language proficiency and possession of regional knowledge.

Evaluating frontier and open-weight models on IINCLUDE-2.0, we find that aggregate multilingual accuracy masks substantial variation across regional knowledge types, languages, and regions. We find that frontier models perform worse on locally embedded knowledge than on widely documented facts about regions, with an average drop of ~14.5 points. We also show that for open-weight models, scale improves accuracy but does not reliably make performance invariant to whether a question is asked in English or its original language. INCLUDE-2.0 is available at https://huggingface.co/datasets/epfl-nlp/include-2.0