I think the actual solution is somewhere in between: If we assume calibrated uncertainty, ignore generalization and assume we can perfectly fit the training data, the total cost should be reduced by (1-the probability assigned to the predicted class) * the cost of misclassifying the not predicted (minority) class as the predicted one (majority): If our classifier already predicted the right class, nothing happens, but otherwise we change our prediction to the other class and reduce the total cost.
While this does not depend on the decision threshold, it does depend on the costs we assign to different misclassifications (in the special case of equal costs, the maximal probability that can be reached by the minority/non-predicted class is 0.5). Edit: This was wrong, the decision threshold is still implicit at 50% in the first paragraph (as cued by the words “majority” and “minority”) : If you apply a 99% decision threshold on a calibrated model, the highest probability you can get for “input is actually unsafe” if your threshold model predicts “safe” is 1%; (now) obviously, you do only get to move examples from predicted “unsafe” to predicted “safe” if you sample close to the 50% threshold, which does not give you much if falsely labelling things as unsafe is not very costly compared to falsely labelling things as safe.
If we however assume that retraining will only shift the prediction probability by epsilon rather than fully flipping the label, we want to minimize the cost from above, subject to only targeting predictions that are epsilon-close to the threshold (as otherwise there won’t be any label flip). In the limit of epsilon->0, we thus should target the prediction threshold rather than 50% (independent of the cost).
In reality, the extent to which predictions will get affected by retraining is certainly more complicated than suggested by these toy models (and we are still only greedily optimizing and completely ignoring generalization). But it might still be useful to think about which of these assumptions seems more realistic.
I think the actual solution is somewhere in between:If we assume calibrated uncertainty, ignore generalization and assume we can perfectly fit the training data, the total cost should be reduced by (1-the probability assigned to the predicted class) * the cost of misclassifying the not predicted (minority) class as the predicted one (majority): If our classifier already predicted the right class, nothing happens, but otherwise we change our prediction to the other class and reduce the total cost.While this does not depend on the decision threshold, it does depend on the costs we assign to different misclassifications (in the special case of equal costs, the maximal probability that can be reached by the minority/non-predicted class is 0.5).Edit: This was wrong, the decision threshold is still implicit at 50% in the first paragraph (as cued by the words “majority” and “minority”) : If you apply a 99% decision threshold on a calibrated model, the highest probability you can get for “input is actually unsafe” if your threshold model predicts “safe” is 1%; (now) obviously, you do only get to move examples from predicted “unsafe” to predicted “safe” if you sample close to the 50% threshold, which does not give you much if falsely labelling things as unsafe is not very costly compared to falsely labelling things as safe.
If we however assume that retraining will only shift the prediction probability by epsilon rather than fully flipping the label, we want to minimize the cost from above, subject to only targeting predictions that are epsilon-close to the threshold (as otherwise there won’t be any label flip). In the limit of epsilon->0, we thus should target the prediction threshold rather than 50% (independent of the cost).
In reality, the extent to which predictions will get affected by retraining is certainly more complicated than suggested by these toy models (and we are still only greedily optimizing and completely ignoring generalization). But it might still be useful to think about which of these assumptions seems more realistic.