Agent utility and utilitarian utility (this renormaization/combining buisness) are two entirely seperate things. No reason the former has to impact the latter, in fact, as we can see, it causes utility monsters and such.
I can’t comment further. Every way I look at it, combining preferences (utilitarianism) is utterly incoherent. Game theory/cooperation seems the only tractible path. I don’t know the context here tho...
if A’s highest preference has no chance of being an outcome then isn’t the solution to fix A’s utility function
Solution for who? A certainly doesn’t want you mucking around it its utility function as that would cause it to not do good things in the universe (from its perspective)
Solution for who? A certainly doesn’t want you mucking around it its utility function as that would cause it to not do good things in the universe (from its perspective)
If A knows that a preferred outcome is completely unobtainable and it knows that some utilitarian theorist is going to discount its preferences with regard to another agent, isn’t it rational to adjust its utility function? Perhaps it’s not; striving for unobtainable goals is somehow a human trait.
In pathological cases like that, sure, you can blackmail it into adjusting its post-op utility function. But only if it became convinced that that gave it a higher chance of getting the things it currently wants.
A lot of those pathological cases go away with reflectively consistent decision thoeries, but perhaps not that one. Don’t feel like working it out.
Agent utility and utilitarian utility (this renormaization/combining buisness) are two entirely seperate things. No reason the former has to impact the latter, in fact, as we can see, it causes utility monsters and such.
I can’t comment further. Every way I look at it, combining preferences (utilitarianism) is utterly incoherent. Game theory/cooperation seems the only tractible path. I don’t know the context here tho...
Solution for who? A certainly doesn’t want you mucking around it its utility function as that would cause it to not do good things in the universe (from its perspective)
If A knows that a preferred outcome is completely unobtainable and it knows that some utilitarian theorist is going to discount its preferences with regard to another agent, isn’t it rational to adjust its utility function? Perhaps it’s not; striving for unobtainable goals is somehow a human trait.
In pathological cases like that, sure, you can blackmail it into adjusting its post-op utility function. But only if it became convinced that that gave it a higher chance of getting the things it currently wants.
A lot of those pathological cases go away with reflectively consistent decision thoeries, but perhaps not that one. Don’t feel like working it out.