Javier Coronado-Blázquez

Author directory

2026

Modern Large Language Models (LLMs) are often criticized for producing repetitive and homogeneous text, despite possessing vast latent vocabularies. While previous research has focused on model knowledge and training data, we investigate the role of decoding mechanics in suppressing linguistic diversity. It is well-known that sampling filters cut the tail of the token probabilities, but the specific impact at the word level remains unexplored. In this paper, we introduce the Word Coverage Score (WCS), a metric that quantifies the extent to which contextually appropriate human vocabulary is mathematically pruned by standard sampling filters (e.g., Top-p, Top-k, and Min-p). Rather than assessing static knowledge, the WCS measures the lexical survival rate of words as a function of sampling parameters through a deterministic Forced-Path Audit. By auditing 13 open-weight models on 10,000 evaluation samples per model from the modern FineWeb corpus at four temperatures, we identify which human lexical choices are rendered unreachable by the decoder, even when they reside within the underlying probability space. To characterize word usage reduction without relying only on absolute erasure, we additionally introduce the Low-Coverage Word Rate (LCWR), the fraction of target words reachable in fewer than a specified proportion of contexts. In matched downstream generation experiments, greater WCS is associated with greater output-text lexical diversity: repeated-measures Spearman correlations are 𝜌=.67 for Type-Token Ratio (TTR), 95% Confidence Interval [.50, .82] and 𝜌 = .66 for Measure of Textual Lexical Diversity (MTLD), 95% Confidence Interval [.51, .81]. These results provide quantitative evidence regarding how inference configurations shape vocabulary compression, establishing WCS as a computationally efficient diagnostic tool for hyperparameter debugging and vocabulary calibration.