When facing an unsatisfactory prediction from a machine learning model, users can be interested in investigating the underlying reasons and exploring the potential for reversing the outcome. We ask: To flip the prediction on a test point x_t, how to identify the smallest training subset 𝒮_t that we need to relabel?We propose an efficient algorithm to identify and relabel such a subset via an extended influence function for binary classification models with convex loss.We find that relabeling fewer than 2% of the training points can always flip a prediction.This mechanism can serve multiple purposes: (1) providing an approach to challenge a model prediction by altering training points; (2) evaluating model robustness with the cardinality of the subset (i.e., |𝒮_t|); we show that |𝒮_t| is highly related to the noise ratio in the training set and |𝒮_t| is correlated with but complementary to predicted probabilities; and (3) revealing training points lead to group attribution bias. To the best of our knowledge, we are the first to investigate identifying and relabeling the minimal training subset required to flip a given prediction.

We consider the problem of identifying a minimal subset of training data 𝒮_t such that if the instances comprising 𝒮_t had been removed prior to training, the categorization of a given test point x_t would have been different.Identifying such a set may be of interest for a few reasons.First, the cardinality of 𝒮_t provides a measure of robustness (if |𝒮_t| is small for x_t, we might be less confident in the corresponding prediction), which we show is correlated with but complementary to predicted probabilities.Second, interrogation of 𝒮_t may provide a novel mechanism for contesting a particular model prediction: If one can make the case that the points in 𝒮_t are wrongly labeled or irrelevant, this may argue for overturning the associated prediction. Identifying 𝒮_t via brute-force is intractable.We propose comparatively fast approximation methods to find 𝒮_t based on influence functions, and find that—for simple convex text classification models—these approaches can often successfully identify relatively small sets of training examples which, if removed, would flip the prediction.