The reason that this sort of works in practice is that information about one member of a reference class is in fact evidence about other members of the reference class. This is just Bayes’ theorem.
Is it? It’s Bayes’ theorem together with the assumption that when you see two things that you’ve put in the same reference class then they’re being drawn from the same distribution. Depending on how the reference class is constructed, this may or may not be a reasonable assumption (if it’s constructed poorly the distribution may have more salient and unknown parameters than you can reasonably learn). At worst, the reference class might be “everything in the universe,” in which case I suppose it’s strictly speaking true that information about one thing in the universe is evidence about other things in the universe, but…
Yes, I wasn’t claiming that it’s good use of Bayes’ theorem. The “sort of” qualification is significant, although I don’t think that the use of the “worst case reference class” prevails in practice :-).
The claim that I intended to make is that “Bayes’ theorem implies that the presence of a feature of a member of a given reference class is evidence for the presence of the feature in other members of the reference class.” This is technically correct. It’s not good epistemology in full generality, for the reason that Qiaochu gives. I’ll modify my post to make what I was trying to say more clear.
Is it? It’s Bayes’ theorem together with the assumption that when you see two things that you’ve put in the same reference class then they’re being drawn from the same distribution. Depending on how the reference class is constructed, this may or may not be a reasonable assumption (if it’s constructed poorly the distribution may have more salient and unknown parameters than you can reasonably learn). At worst, the reference class might be “everything in the universe,” in which case I suppose it’s strictly speaking true that information about one thing in the universe is evidence about other things in the universe, but…
Yes, I wasn’t claiming that it’s good use of Bayes’ theorem. The “sort of” qualification is significant, although I don’t think that the use of the “worst case reference class” prevails in practice :-).
I think most readers will read the phrase “This is just Bayes’ theorem” as “This is correct use of Bayes’ theorem.”
The claim that I intended to make is that “Bayes’ theorem implies that the presence of a feature of a member of a given reference class is evidence for the presence of the feature in other members of the reference class.” This is technically correct. It’s not good epistemology in full generality, for the reason that Qiaochu gives. I’ll modify my post to make what I was trying to say more clear.