If both the agent and Omega are deterministic programs, and the agent is never in fact asked, that fact may be converted into a statement about natural numbers. So what you just said is equivalent to this:
Seems to me that for all agents there is a fact of the matter about whether they would pay if 1 were equal to 2.
Why? Say the world program W includes function f, and it’s provable that W could never call f with argument 1. That doesn’t mean there’s no fact of the matter about what happens when f(1) is computed (though of course it might not halt). (Function f doesn’t have to be called from W.)
Even if f can be regarded as a rational agent who ‘knows’ the source code of W, the worst that could happen is that f ‘deduces’ a contradiction and goes insane. That’s different from the agent itself being in an inconsistent state.
Analogy: We can define the partial derivatives of a Lagrangian with respect to q and q-dot, even though it doesn’t make sense for q and q-dot to vary independently of each other.
If both the agent and Omega are deterministic programs, and the agent is never in fact asked, that fact may be converted into a statement about natural numbers. So what you just said is equivalent to this:
I don’t know, this looks shady.
Why? Say the world program W includes function f, and it’s provable that W could never call f with argument 1. That doesn’t mean there’s no fact of the matter about what happens when f(1) is computed (though of course it might not halt). (Function f doesn’t have to be called from W.)
Even if f can be regarded as a rational agent who ‘knows’ the source code of W, the worst that could happen is that f ‘deduces’ a contradiction and goes insane. That’s different from the agent itself being in an inconsistent state.
Analogy: We can define the partial derivatives of a Lagrangian with respect to q and q-dot, even though it doesn’t make sense for q and q-dot to vary independently of each other.