Indexical values are not reflectively consistent. UDT “solves” this problem by implicitly assuming (via the type signature of its utility function) that the agent doesn’t have indexical values. But humans seemingly do have indexical values, so what to do about that?
Your linked post seems to suggest only some varieties of indexical value sources are reflectively inconsistent, but what’s missing is an indexical value source that’s both reflectively consistent and makes sense for humans. So there could still be a way to make indexical values reflectively consistent, just that we haven’t thought of it yet?
E.g. would it work to privilege an agent’s original instantiation, so that if you’re uncertain you’re the original or the copy you follow the interests of the original? That would seem to address the counterfactual mugging question if Omega were to predict by simulating you at least.
(I’m not sure if that technically counts as ‘indexical’ but it seems to me it can still be ‘selfish’ in the colloquial sense, no?)
You can use it when it can be well defined. I think in the real world you mostly do have something at least in the past you can call “original”, and when it doesn’t still exist you could modify to, e.g. “what the original instantiation, if it anticipated this scenario, would have defined as its successor”.
Your linked post seems to suggest only some varieties of indexical value sources are reflectively inconsistent, but what’s missing is an indexical value source that’s both reflectively consistent and makes sense for humans. So there could still be a way to make indexical values reflectively consistent, just that we haven’t thought of it yet?
E.g. would it work to privilege an agent’s original instantiation, so that if you’re uncertain you’re the original or the copy you follow the interests of the original? That would seem to address the counterfactual mugging question if Omega were to predict by simulating you at least.
(I’m not sure if that technically counts as ‘indexical’ but it seems to me it can still be ‘selfish’ in the colloquial sense, no?)
I’m not sure “original instantiation” is always well-defined
You can use it when it can be well defined. I think in the real world you mostly do have something at least in the past you can call “original”, and when it doesn’t still exist you could modify to, e.g. “what the original instantiation, if it anticipated this scenario, would have defined as its successor”.