Any non-uniform prior inherently encodes a bias toward simplicity. This isn’t an additional assumption we need to make—it falls directly out of the mathematics.
For any hypothesis h, the information content is I(h)=−log(P(h)), which means probability and complexity have an exponential relationship: P(h)=e−I(h)
This demonstrates that simpler hypotheses (those with lower information content) are automatically assigned higher probabilities. The exponential relationship creates a strong bias toward simplicity without requiring any special mechanisms.
The “simplicity prior” is essentially tautological—more probable things are simple by definition.
You can have a hypothesis with really high kolmogorov complexity, but if the hypothesis is true 50% of the time it will require 1 bit of information to specify with respect to a coding scheme that merely points to cached hypotheses.
This is why when kolmogorov complexity is defined it’s with respect to a fixed universal description language, as otherwise you’re right, it’s vacuous to talk about the simplicity of a hypothesis.
Simplicity Priors are Tautological
Any non-uniform prior inherently encodes a bias toward simplicity. This isn’t an additional assumption we need to make—it falls directly out of the mathematics.
For any hypothesis h, the information content is I(h)=−log(P(h)), which means probability and complexity have an exponential relationship: P(h)=e−I(h)
This demonstrates that simpler hypotheses (those with lower information content) are automatically assigned higher probabilities. The exponential relationship creates a strong bias toward simplicity without requiring any special mechanisms.
The “simplicity prior” is essentially tautological—more probable things are simple by definition.
You can have a hypothesis with really high kolmogorov complexity, but if the hypothesis is true 50% of the time it will require 1 bit of information to specify with respect to a coding scheme that merely points to cached hypotheses.
This is why when kolmogorov complexity is defined it’s with respect to a fixed universal description language, as otherwise you’re right, it’s vacuous to talk about the simplicity of a hypothesis.