Solomonoff induction is generally given as the correct way to penalise more complex hypotheses when calculating priors. A great introduction can be found here.
My question is, how is this actually calculated in practice?
As an example, say I have 2 hypotheses:
A. The probability distribution of the output is given by the same normal distribution for all inputs, with mean μ and standard deviation σ.
B. The probability distribution of the output is given by a normal distribution depending on an input x with mean μ0+mx and standard deviation σ.
It is clear that hypothesis B is more complex (using an additional input [x], having an additional parameter [m] and requiring 2 additional operations to calculate) but how does one calculate the actual penalty that B should be given vs A?
[Question] How is Solomonoff induction calculated in practice?
Solomonoff induction is generally given as the correct way to penalise more complex hypotheses when calculating priors. A great introduction can be found here.
My question is, how is this actually calculated in practice?
As an example, say I have 2 hypotheses:
A. The probability distribution of the output is given by the same normal distribution for all inputs, with mean μ and standard deviation σ.
B. The probability distribution of the output is given by a normal distribution depending on an input x with mean μ0+mx and standard deviation σ.
It is clear that hypothesis B is more complex (using an additional input [x], having an additional parameter [m] and requiring 2 additional operations to calculate) but how does one calculate the actual penalty that B should be given vs A?