I suspect Eliezer would object to my post claiming that I’m confusing map and territory, but I don’t think that’s fair. If there’s a map you’re trying to use all over the place (and you do seem to), then I claim it makes no sense to put a little region on the map labelled “maybe this map doesn’t make any sense at all”. If the map seems to make sense and you’re still following it for everything, you’ll have to ignore that region anyway. So is it really reasonable to claim that “the probability that probability makes sense is <1″?
Utilitarian:
Measure theory gives a clear answer to this: it’s 0. Which is fine. For all x, the probability that your rv will take the value x is 0. Actually the probability that your rv is computable is also 0. (Computable numbers are the largest countable class I know of.) What’s false is the tempting statement that probability 0 events are impossible. It’s only the converse that’s true: impossible events have probability 0. There’s another tempting statement that’s false, namely the statement that if S is an arbitrary collection of disjoint events, the probability of one of them happening is the sum of the probabilities of each one happening. Instead, this only holds for countable sets S. This is part of the definition of a measure.
I agree with cumulant. The mathematical subject of probability is based on measure theory, which loses a ton of convergence theorems if we exclude 0 and 1. We can agree that things that are not known a priori can’t have probability 0 or 1, but I think we must also agree that “an impossible thing will happen soon” has probability 0, because it’s a contradiction. An alternate universe in which the number 7 (in the same kind of number system as ours, etc.) is prime is damn-near inconceivable, but an alternate universe in which impossible things are possible is purely absurd.
If our mathematical reasoning is coherent enough for it to be meaningful to make probability assignments then certainly we are not so fundamentally flawed that what we consider tautologies could be false. If you are willing to accept that maybe 0 is 1, then you can’t do any of your probability adjustments, or use Bayes’ Theorem, or anything of the sort without having a (possibly unstated) caveat that probability theory might be complete nonsense. But what’s the probability that probability theory is nonsense (i.e. false or inconsistent)? What does that even mean? We can only assign a probability if that makes sense, so conditioned on the sentence making sense, probability theory must be nonsense with probability 0, no? So averaged over all possible universes (those where probability theory makes sense, and those where it doesn’t) the sentence “probability makes sense with probability 1” better approximates the truth value of probability making sense than “probability makes sense with probability p” for p0. If it’s not, it’s still not worse, but what the hell are we even saying?