This is a real elephant in the room. It’s been mentioned a few times here, but it remains a major epidiment to Bayes is the Only Epistemology you need, and other cherished notions.
The problem of ignored hypotheses with known relations
The biggest problem is with the “hypotheses and logical relations” setup.
The setup is deceptively easy to use in toy problems where you can actually list all of the possible hypotheses. The classic example is a single roll of a fair six-sided die. There is a finite list of distinct hypotheses one could have about the outcome, and they are all generated by conjunction/disjunction of the six “smallest” hypotheses, which assert that the die will land on one specific face. Using the set formalism, we can write these as
{1}, {2}, {3}, {4}, {5}, {6}
Any other hypothesis you can have is just a set with some of these numbers in it. “2 or 5″ is {2, 5}. “Less than 3” is just {1, 2}, and is equivalent to “1 or 2.” “Odd number” is {1, 3, 5}.
Since we know the specific faces are mutually exclusive and exhaustive, and we know their probabilities (all 1⁄6), it’s easy to compute the probability of any other hypothesis: just count the number of elements. {2, 5} has probability 2⁄6, and so forth. Conditional probabilities are easy too: conditioning on “odd number” means the possible faces are {1, 3, 5}, so now {2, 5} has conditional probability 1⁄3, because only one of the three possibilities is in there.
Because we were building sets out of individual members, here, we automatically obeyed the logical consistency rules, like not assigning “A or B” a smaller probability than “A.” We assigned probability 2⁄6 to “2 or 5″ and probability 1⁄6 to “2,” but we didn’t do that by thinking “hmm, gotta make sure we follow the consistency rules.” We could compute the probabilities exactly from first principles, and of course they followed the rules.
In most real-world cases of interest, though, we are not building up hypotheses from atomic outcomes in this exact way. Doing that is equivalent to stating exact necessary and sufficient conditions in terms of the finest-grained events we can possibly imagine; to do it for a hypothesis like “Trump will be re-elected in 2020,” we’d have to write down all the possible worlds where Trump wins, and the ones where he doesn’t, in terms of subatomic physics.
Instead, what we have in the real world is usually a vast multiple of conceivable hypotheses, very few of which we have actively considered (or will ever consider), and – here’s the kicker – many of these unconsidered hypotheses have logical relations to the hypotheses under consideration which we’d know if we considered them.
Thanks for pointing out that post by nostalgebraist. I had not seen it before and it definitely is of interest to me. I’m interested in hearing anything else along these lines, particularly information about solving this problem.
This is a real elephant in the room. It’s been mentioned a few times here, but it remains a major epidiment to Bayes is the Only Epistemology you need, and other cherished notions.
http://nostalgebraist.tumblr.com/post/161645122124/bayes-a-kinda-sorta-masterpost
Thanks for pointing out that post by nostalgebraist. I had not seen it before and it definitely is of interest to me. I’m interested in hearing anything else along these lines, particularly information about solving this problem.