I outlined the problem with mathematical uncertainty here.
Don’t agree with the premise there. As to what Yudkowsky is talking about by saying “Logical uncertainty is an open problem”—it beats me. There’s really only uncertainty. Uncertainty about mathematics is much the same as other kinds of uncertainty.
I can prove that to you, unless I made a mistake. Are you saying you can defeat it a priori by telling me a prior that doesn’t have any of those three properties?
Take induction, for example, where the domain of the function P(X) ranges over the possible symbols that might come next in the stream (or, if you prefer, ranges over the hypotheses that predict them).
Then P(X and Y) is typically not a meaningful concept.
It is not defined on all X—i.e. P is agnostic about some things
It has P(X) < P(X and Y) for at least one pair X and Y—i.e. P sometimes falls for the conjunction fallacy
It has P(X) = 1 for all mathematically provable statements X—i.e. P is an oracle.
Taken literally, your P falls for 1. For instance it doesn’t have an opinion about whether it will be sunny tomorrow or the Riemann hypothesis is true. If you wish avoid this problem by encoding the universe as a string of symbols to feed to your induction machine, why wouldn’t you let me encode “X and Y” at the same time?
Don’t agree with the premise there. As to what Yudkowsky is talking about by saying “Logical uncertainty is an open problem”—it beats me. There’s really only uncertainty. Uncertainty about mathematics is much the same as other kinds of uncertainty.
What premise?
The first 5 lines of the post—this bit::
I can prove that to you, unless I made a mistake. Are you saying you can defeat it a priori by telling me a prior that doesn’t have any of those three properties?
Take induction, for example, where the domain of the function P(X) ranges over the possible symbols that might come next in the stream (or, if you prefer, ranges over the hypotheses that predict them).
Then P(X and Y) is typically not a meaningful concept.
The trichotomy is:
Taken literally, your P falls for 1. For instance it doesn’t have an opinion about whether it will be sunny tomorrow or the Riemann hypothesis is true. If you wish avoid this problem by encoding the universe as a string of symbols to feed to your induction machine, why wouldn’t you let me encode “X and Y” at the same time?