To speak of Knightian uncertainty we need a way of separating our uncertainty into “Knightian” and “Bayesian” components. There is actually a natural candidate for that. Namely, Bayesian uncertainty comes from a Solomonoff ensemble and Knightian uncertainty comes from our limited ability to compute the Solomonoff expectation value. For example if the agent is reasoning within a certain formal system, this system will generally allow proving “a ⇐ E(U) ⇐ b” for some a and b but the bounds won’t coincide in non-trivial cases because of the halting problem. We might have hope that a theory of logical uncertainty would allow producing a crisp expectation value but I’m not so sure. My feeling is that “logical uncertainty” (i.e. Bayesian reasoning with limited computing resources) only works well for computable sentences / quantities (see e.g. my attempt to define it) whereas for uncomputable quantities we need to use a sequence of computable approximations. In the case of the Solomonoff ensemble the natural computable approximations seem to be imposing cutoffs on the running time of the programs within the ensemble. This, however, means we need to bundle together programs that produce definite predictions for all times with programs that produce definite predictions for a given physical time span and fail to halt later. In particular, we need a way to assign utilities to partial possible histories. It seems to me that the natural way to do it would be combining time discount with worst-case assumptions regarding the undefined future, so that problems like the procrastination paradox are avoided. These worst-case assumptions looks quite like the MMEU rule.
This said, I’m not sure this “pure” Knightian uncertainty explains most ambiguity aversion in real life scenarios: the latter might still be a bias.
I mostly agree with this, though I’m not yet sold on assigning utilities to partial histories using worst-case assumptions. What exactly do you mean by combining partial histories with worst-case assumptions? How is ‘worst case’ defined?
For every partial history, there is a turing machine which turns it into a hellscape in the “undefined future.” This comes with a huge complexity penalty, of course, but these are precisely the sorts of things you need to watch out for when you start maximizing worst case scenarios (which neglect complexity penalties) rather than likely scenarios.
In order to avoid the procrastination paradox, we want our utility function to be upper semicontinuous in the natural topology on histories. Given such a utility function U on the space of infinite histories, there is a natural way to extend it to the space of finite & infinite histories preserving semicontinuity. Namely, we define U(x) = inf U(xy) where x is a finite history and y is an infinite continuation.
However, this prescription is not necessary: we can have an upper semicontinuous function in the space of finite & infinite histories which doesn’t arise in this way. Coming to think about it, it isn’t very attractive since intuitively the universe coming to end is a better outcome than the universe turning into hell.
To speak of Knightian uncertainty we need a way of separating our uncertainty into “Knightian” and “Bayesian” components. There is actually a natural candidate for that. Namely, Bayesian uncertainty comes from a Solomonoff ensemble and Knightian uncertainty comes from our limited ability to compute the Solomonoff expectation value. For example if the agent is reasoning within a certain formal system, this system will generally allow proving “a ⇐ E(U) ⇐ b” for some a and b but the bounds won’t coincide in non-trivial cases because of the halting problem. We might have hope that a theory of logical uncertainty would allow producing a crisp expectation value but I’m not so sure. My feeling is that “logical uncertainty” (i.e. Bayesian reasoning with limited computing resources) only works well for computable sentences / quantities (see e.g. my attempt to define it) whereas for uncomputable quantities we need to use a sequence of computable approximations. In the case of the Solomonoff ensemble the natural computable approximations seem to be imposing cutoffs on the running time of the programs within the ensemble. This, however, means we need to bundle together programs that produce definite predictions for all times with programs that produce definite predictions for a given physical time span and fail to halt later. In particular, we need a way to assign utilities to partial possible histories. It seems to me that the natural way to do it would be combining time discount with worst-case assumptions regarding the undefined future, so that problems like the procrastination paradox are avoided. These worst-case assumptions looks quite like the MMEU rule.
This said, I’m not sure this “pure” Knightian uncertainty explains most ambiguity aversion in real life scenarios: the latter might still be a bias.
I mostly agree with this, though I’m not yet sold on assigning utilities to partial histories using worst-case assumptions. What exactly do you mean by combining partial histories with worst-case assumptions? How is ‘worst case’ defined?
For every partial history, there is a turing machine which turns it into a hellscape in the “undefined future.” This comes with a huge complexity penalty, of course, but these are precisely the sorts of things you need to watch out for when you start maximizing worst case scenarios (which neglect complexity penalties) rather than likely scenarios.
In order to avoid the procrastination paradox, we want our utility function to be upper semicontinuous in the natural topology on histories. Given such a utility function U on the space of infinite histories, there is a natural way to extend it to the space of finite & infinite histories preserving semicontinuity. Namely, we define U(x) = inf U(xy) where x is a finite history and y is an infinite continuation.
However, this prescription is not necessary: we can have an upper semicontinuous function in the space of finite & infinite histories which doesn’t arise in this way. Coming to think about it, it isn’t very attractive since intuitively the universe coming to end is a better outcome than the universe turning into hell.