Clustering is often a process of trial and error: we apply different algorithms (such as k-means or hdbscan), apply different embeddings, try a dimensionality reduction, explore different values of K, inspect the result with the product manager, and repeat. When the data is large, even communicating and analyzing a single clustering becomes difficult.

We argue that the nature of clustering requires a fundamentally different approach. We posit that clustering is fundamentally a subjective and iterative process, and its central challenges are how to elicit the dimensions and nuances a user cares about, how to communicate a clustering effectively, and how to converge to an output of interest through efficient conversation. We reframe clustering as a conversational supervision problem. We formalize conversational clustering as a new task, derive evaluation metrics suited to it, and release an evaluation framework and reference implementation to seed further work.