Topographic Similarity (TopSim) is the most widely used metric for compositionality in emergent communication research. However, when evaluated on a class of languages where compositional structure is fully known (simple natural-language word concatenations), TopSim, along with other well-known metrics, under-reports compositionality. We investigate and trace TopSim’s failure to a distributional imbalance in pairwise object distances that grows with the input space size, and propose stratified-TopSim, a minimal fix that remains stable across settings and reliably evaluates languages of varying compositional structure, on both controlled benchmarks and emergent languages.

In natural languages, frequency and compositionality exhibit an inverse relationship: the most frequent words often resist regular patterns, developing idiosyncratic forms. This phenomenon, exemplified by irregular verbs where the most frequent verbs resist regular patterns, raises a compelling question: do artificial communication systems follow similar principles?Through systematic experiments with neural network agents in a referential game setting, and by manipulating input frequency through Zipfian distributions, we investigate if these systems mirror the irregular verbs phenomenon, where messages referring to frequent objects develop less compositional structure than messages referring to rare ones.We establish that compositionality is not an inherent property of the frequency itself and provide compelling evidence that limited data exposure, which frequency distributions naturally create, serves as a fundamental driver for the emergence of compositional structure in communication systems, offering insights into the cognitive and computational pressures that shape linguistic systems.