MIRI had a similar question, and thus was born logical induction. If you think of probabilities on logical statements as prices of a prediction market, and you require that no polynomial time trader can exploit the market (meaning unbounded gains for finite upfront investment) then you’ll have sensible properties like “the gabillionth digit of pi has a 1⁄10 chance of being any specific digit”, and you can actually construct such an inexploitable market.
In this case, there’s not really any problem—the probabilities are on logical statements, not on random samples of reals (though if you write the question as a logical statement, you’ll be able to put a probability on it).
MIRI had a similar question, and thus was born logical induction. If you think of probabilities on logical statements as prices of a prediction market, and you require that no polynomial time trader can exploit the market (meaning unbounded gains for finite upfront investment) then you’ll have sensible properties like “the gabillionth digit of pi has a 1⁄10 chance of being any specific digit”, and you can actually construct such an inexploitable market.
In this case, there’s not really any problem—the probabilities are on logical statements, not on random samples of reals (though if you write the question as a logical statement, you’ll be able to put a probability on it).