you can train on MNIST digits with twenty wrong labels for every correct one and still get good performance as long as the correct label is slightly more common than the most common wrong label
Simplicia: Oh! Because if there are nine wrong labels that aren’t individually more common than the correct label, then the most they can collectively outnumber the correct label is by 9 to 1. But I could have sworn that Rolnick et al. §3.2 said that—oh, I see. I misinterpreted Figure 4. I should have said “twenty noisy labels for every correct one”, not “twenty wrong labels”—where some of the noisy labels are correct “by chance”.
For example, training examples with the correct label 0 could appear with the label 0 for sure 10 times, and then get a uniform random label 200 times, and thus be correctly labeled 10 + 200⁄10 = 30 times, compared to 20 for each wrong label. (In expectation—but you also could set it up so that the “noisy” labels don’t deviate from the expected frequencies.) That doesn’t violate the pigeonhole principle.
I regret the error. Can we just—pretend I said the correct thing? If there were a transcript of what I said, it would only be a one-word edit. Thanks.
I know some pigeons who would question this claim
Simplicia: Oh! Because if there are nine wrong labels that aren’t individually more common than the correct label, then the most they can collectively outnumber the correct label is by 9 to 1. But I could have sworn that Rolnick et al. §3.2 said that—oh, I see. I misinterpreted Figure 4. I should have said “twenty noisy labels for every correct one”, not “twenty wrong labels”—where some of the noisy labels are correct “by chance”.
For example, training examples with the correct label
0could appear with the label0for sure 10 times, and then get a uniform random label 200 times, and thus be correctly labeled 10 + 200⁄10 = 30 times, compared to 20 for each wrong label. (In expectation—but you also could set it up so that the “noisy” labels don’t deviate from the expected frequencies.) That doesn’t violate the pigeonhole principle.I regret the error. Can we just—pretend I said the correct thing? If there were a transcript of what I said, it would only be a one-word edit. Thanks.