Chinese classifier prediction is often evaluated as a single-label classification problem, assuming one gold classifier for each context. We argue that this assumption overlooks the inherent variability and conceptual nature of classifier use. We propose a controlled experimental framework for studying Chinese classifiers in humans and vision-language models across three tasks: Classifier Choice, Head Noun Completion, and Referring Expression Generation. Based on a human behavioural dataset, we evaluate models against human response distributions rather than single gold labels. Our results show that classifier use is shaped by noun specificity, animacy, and task setting, and that models differ from humans not only in classifier choice but also in classifier-mediated conceptualisation. Although explicitly encoding the behavioural patterns learned from the human experiments into prompts improves model alignment with human behaviour, especially for smaller models, its effect is uneven across tasks: it helps models better match dominant human preferences, but does not enable them to reproduce the full distribution of human variability or fully close the gap in natural generation settings.