I actually have a different idea on a possible approach to this Problem. I tried to get a discussion about it started in the decision theory mailing group, but it sort of died, and I’m not sure it’s safe to post in public.
Don’t be silly; it’s completely safe.
Q: Under what circumstances is an agent with utility function U isomorphic to one with utility function U’, where if U equals negative infinity U’=-1, if U is any finite real number U’=0, and if U equals positive infinity U’=1?
The original agent, with a utility function that sometimes kicks out infinities, is undefined; those infinities can’t be compared, and they propagate into the expected utility of actions so that actions can’t be compared either. The replacement agent is defined, but only has a three-valued utility function, which means it can’t express many preferences.
You get the equivalence you want if: your utilities lie in a totally ordered field extension of R, infinity is a constant greater than all elements of R, the utility of pure outcomes are restricted to be either R or +- infinity, and the relationship between utility and decision is in a certain generic position (so that the probability of every outcome changes whenever you change your decision, and these changes are never arranged to exactly make the utilities cancel out).
Don’t be silly; it’s completely safe.
The original agent, with a utility function that sometimes kicks out infinities, is undefined; those infinities can’t be compared, and they propagate into the expected utility of actions so that actions can’t be compared either. The replacement agent is defined, but only has a three-valued utility function, which means it can’t express many preferences.
Always, independently of what kind of agent and what kind of infinities you use?
You get the equivalence you want if: your utilities lie in a totally ordered field extension of R, infinity is a constant greater than all elements of R, the utility of pure outcomes are restricted to be either R or +- infinity, and the relationship between utility and decision is in a certain generic position (so that the probability of every outcome changes whenever you change your decision, and these changes are never arranged to exactly make the utilities cancel out).
I’m not sure I understood all of that, but the pieces I did sound likely to be true about CEV or a papperclipper to me. Am I missing something?