You should want to label and train on snippets that your classifier thinks is 50% correct, because that is how you maximmise information.
You don’t want to ‘maximize information’ (or minimize variance). You want to minimize the number of errors you make at your decision-threshold. Your threshold is not at 50%, it’s at 99%. Moving an evil sample from 50% to 0% is of zero intrinsic value (because you have changed the decision from ‘Reject’ to ‘Reject’ and avoided 0 errors). Moving an evil sample from 99.1% to 98.9% is very valuable (because you have changed the decision from ‘Accept’ to ‘Reject’ and avoided 1 error). Reducing the error on regions of data vastly far away from the decision threshold, such as deciding whether a description of a knifing is ever so slightly more ‘violent’ than a description of a shooting and should be 50.1% while the shooting is actually 49.9%, is an undesirable use of labeling time.
The correct labeling of how violent a knifing is, is not 50.1%, or 49.9%. The correct label is 0 or 100%. There is no “ever so slightly” in the training data. The percentage is about the uncertanty of classifyer, it is not about degrees of violence in the sample. It it was the other way around, then I would mostsy agree with the current training scheem, as I said.
If the model is well calibrated then half the samples would be safe, and half violent at 50%. Moving a up the safe one is helpfull. Decreesing missclassification of safe samples will increas the chance of outputing something safe.
Decreesing the uncertanty from 50% to 0 for an unsafe sample don’t do anything, for that sample. But it does help in learning good from bad in general, which is more important.
I think the actual solution is somewhere in between: If we assume calibrated uncertainty, ignore generalization and assume we can perfectly fit the training data, the total cost should be reduced by (1-the probability assigned to the predicted class) * the cost of misclassifying the not predicted (minority) class as the predicted one (majority): If our classifier already predicted the right class, nothing happens, but otherwise we change our prediction to the other class and reduce the total cost.
While this does not depend on the decision threshold, it does depend on the costs we assign to different misclassifications (in the special case of equal costs, the maximal probability that can be reached by the minority/non-predicted class is 0.5). Edit: This was wrong, the decision threshold is still implicit at 50% in the first paragraph (as cued by the words “majority” and “minority”) : If you apply a 99% decision threshold on a calibrated model, the highest probability you can get for “input is actually unsafe” if your threshold model predicts “safe” is 1%; (now) obviously, you do only get to move examples from predicted “unsafe” to predicted “safe” if you sample close to the 50% threshold, which does not give you much if falsely labelling things as unsafe is not very costly compared to falsely labelling things as safe.
If we however assume that retraining will only shift the prediction probability by epsilon rather than fully flipping the label, we want to minimize the cost from above, subject to only targeting predictions that are epsilon-close to the threshold (as otherwise there won’t be any label flip). In the limit of epsilon->0, we thus should target the prediction threshold rather than 50% (independent of the cost).
In reality, the extent to which predictions will get affected by retraining is certainly more complicated than suggested by these toy models (and we are still only greedily optimizing and completely ignoring generalization). But it might still be useful to think about which of these assumptions seems more realistic.
You don’t want to ‘maximize information’ (or minimize variance). You want to minimize the number of errors you make at your decision-threshold. Your threshold is not at 50%, it’s at 99%. Moving an evil sample from 50% to 0% is of zero intrinsic value (because you have changed the decision from ‘Reject’ to ‘Reject’ and avoided 0 errors). Moving an evil sample from 99.1% to 98.9% is very valuable (because you have changed the decision from ‘Accept’ to ‘Reject’ and avoided 1 error). Reducing the error on regions of data vastly far away from the decision threshold, such as deciding whether a description of a knifing is ever so slightly more ‘violent’ than a description of a shooting and should be 50.1% while the shooting is actually 49.9%, is an undesirable use of labeling time.
The correct labeling of how violent a knifing is, is not 50.1%, or 49.9%. The correct label is 0 or 100%. There is no “ever so slightly” in the training data. The percentage is about the uncertanty of classifyer, it is not about degrees of violence in the sample. It it was the other way around, then I would mostsy agree with the current training scheem, as I said.
If the model is well calibrated then half the samples would be safe, and half violent at 50%. Moving a up the safe one is helpfull. Decreesing missclassification of safe samples will increas the chance of outputing something safe.
Decreesing the uncertanty from 50% to 0 for an unsafe sample don’t do anything, for that sample. But it does help in learning good from bad in general, which is more important.
I think the actual solution is somewhere in between:If we assume calibrated uncertainty, ignore generalization and assume we can perfectly fit the training data, the total cost should be reduced by (1-the probability assigned to the predicted class) * the cost of misclassifying the not predicted (minority) class as the predicted one (majority): If our classifier already predicted the right class, nothing happens, but otherwise we change our prediction to the other class and reduce the total cost.While this does not depend on the decision threshold, it does depend on the costs we assign to different misclassifications (in the special case of equal costs, the maximal probability that can be reached by the minority/non-predicted class is 0.5).Edit: This was wrong, the decision threshold is still implicit at 50% in the first paragraph (as cued by the words “majority” and “minority”) : If you apply a 99% decision threshold on a calibrated model, the highest probability you can get for “input is actually unsafe” if your threshold model predicts “safe” is 1%; (now) obviously, you do only get to move examples from predicted “unsafe” to predicted “safe” if you sample close to the 50% threshold, which does not give you much if falsely labelling things as unsafe is not very costly compared to falsely labelling things as safe.
If we however assume that retraining will only shift the prediction probability by epsilon rather than fully flipping the label, we want to minimize the cost from above, subject to only targeting predictions that are epsilon-close to the threshold (as otherwise there won’t be any label flip). In the limit of epsilon->0, we thus should target the prediction threshold rather than 50% (independent of the cost).
In reality, the extent to which predictions will get affected by retraining is certainly more complicated than suggested by these toy models (and we are still only greedily optimizing and completely ignoring generalization). But it might still be useful to think about which of these assumptions seems more realistic.