Nice post! I hadn’t read the previous posts on Cake or Death-like problems, but your post persuaded me to read them.
Enforcing indifference between choices to prevent abuse is a beautiful idea. You remark that as a consequence of this indifference the expected change in utility for changing from utility function ‘v’ to function ‘w’ is equal to 0, but isn’t the converse also true (i.e. the desired update formula follows from the conservation of expected utility)? Above you justify/motivate the change in the update formula (marked in bold in your post) to E(w|v->w) on grounds of symmetry, but doesn’t it simply follow from demanding that changing the utility function itself should not change the (average) utiliy? This sounds like a direct formalisation of demanding that our agent is indifferent with respect to its own utility function.
Nice post! I hadn’t read the previous posts on Cake or Death-like problems, but your post persuaded me to read them.
Enforcing indifference between choices to prevent abuse is a beautiful idea. You remark that as a consequence of this indifference the expected change in utility for changing from utility function ‘v’ to function ‘w’ is equal to 0, but isn’t the converse also true (i.e. the desired update formula follows from the conservation of expected utility)? Above you justify/motivate the change in the update formula (marked in bold in your post) to E(w|v->w) on grounds of symmetry, but doesn’t it simply follow from demanding that changing the utility function itself should not change the (average) utiliy? This sounds like a direct formalisation of demanding that our agent is indifferent with respect to its own utility function.
I think this is correct. When I write the principle up properly, I’ll probably be emphasising that issue more.