First of all, I’d like to avoid just specifying by fiat that simpler hypotheses get higher prior probability and instead have this be a consequence of more solid principles. I think the principle of indifference is solid, if we can find a privileged parameterization of the hypothesis space to which we can apply the principle.
Solomonoff induction (SI) is already reducible to the principle of indifference. For UTMs with prefix-free codes, SI assigns prior 2−K to programs of length K.[1] But there’s another formulation over plain, non-prefix free codes, where you just take the uniform prior over all programs of length T, then let T go to infinity. These are equivalent. See this book on pages 145-146.
If this is unintuitive to you, I’d suggest having a look at this shortform or this one. The multiplicity of equivalent implementations automatically encodes the exponentially larger prior for simpler programs.
I think the only thing left to debate over here is why we should start out assuming that our observations of the world are produced by some program running on some UTM, rather than, say, assigning equal probability to all strings of bits we might observe. In what sense does the universe have structure, such that a-priori bit strings of observations about the universe ought to be treated by us as more than just members of the set of all possible bit strings?
rather than, say, assigning equal probability to all strings of bits we might observe
If the space of possibilities is not arbitrarily capped at a certain length, then such a distribution would have to favor shorter strings over longer ones in much the same way as the Solomonoff prior over programs (because if it doesn’t, then its sum will diverge, etc.). But then this yields a prior that is constantly predicting that the universe will end at every moment, and is continually surprised when it keeps on existing. I’m not sure if this is logically inconsistent, but at least it seems useless for any practical purpose.
where you just take the uniform prior over all programs of length T, then let T to infinity
Sure, but because of language-dependence I’m not sure why we would want to apply the principle of indifference at this level. (Note that the quote says “if we can find a privileged parameterization to which we can apply the principle”.) I tend to think you should apply the POI at the “explanatorily basic” level (see here, here), which might be the properties of fundamental objects in the ontology (e.g., position and momentum in Newtonian mechanics, maybe?). Otherwise I think you run into unsatisfying-to-me arbitrariness.
In what sense does the universe have structure, such that a-priori bit strings of observations about the universe ought to be treated by us as more than members of the set of all possible bit strings?
Right, I think this is the kind of question you’re not going to be able to answer without thinking about the kinds of ontological considerations pointed to here.
Solomonoff induction (SI) is already reducible to the principle of indifference. For UTMs with prefix-free codes, SI assigns prior 2−K to programs of length K.[1] But there’s another formulation over plain, non-prefix free codes, where you just take the uniform prior over all programs of length T, then let T go to infinity. These are equivalent. See this book on pages 145-146.
If this is unintuitive to you, I’d suggest having a look at this shortform or this one. The multiplicity of equivalent implementations automatically encodes the exponentially larger prior for simpler programs.
I think the only thing left to debate over here is why we should start out assuming that our observations of the world are produced by some program running on some UTM, rather than, say, assigning equal probability to all strings of bits we might observe. In what sense does the universe have structure, such that a-priori bit strings of observations about the universe ought to be treated by us as more than just members of the set of all possible bit strings?
This is not the same as giving hypothesis h 2−C(p,L) prior as you state near the beginning, afaik that’s false.
If the space of possibilities is not arbitrarily capped at a certain length, then such a distribution would have to favor shorter strings over longer ones in much the same way as the Solomonoff prior over programs (because if it doesn’t, then its sum will diverge, etc.). But then this yields a prior that is constantly predicting that the universe will end at every moment, and is continually surprised when it keeps on existing. I’m not sure if this is logically inconsistent, but at least it seems useless for any practical purpose.
Thanks!
Sure, but because of language-dependence I’m not sure why we would want to apply the principle of indifference at this level. (Note that the quote says “if we can find a privileged parameterization to which we can apply the principle”.) I tend to think you should apply the POI at the “explanatorily basic” level (see here, here), which might be the properties of fundamental objects in the ontology (e.g., position and momentum in Newtonian mechanics, maybe?). Otherwise I think you run into unsatisfying-to-me arbitrariness.
Right, I think this is the kind of question you’re not going to be able to answer without thinking about the kinds of ontological considerations pointed to here.
Thanks, will change.