Why don’t you have to take into account the prior probability of the large mistake that occurred? Of course, you might be biased and believe it to be smaller than it truly is, in which case there should be a whoops moment (your mistake was overconfidence), but clearly there must also be cases in which there was a small prior probability of a big mistake. Shouldn’t we not judge these cases by only examining the outcome?
Why don’t you have to take into account the prior probability of the large mistake that occurred? Of course, you might be biased and believe it to be smaller than it truly is, in which case there should be a whoops moment (your mistake was overconfidence), but clearly there must also be cases in which there was a small prior probability of a big mistake. Shouldn’t we not judge these cases by only examining the outcome?