“Not to my benefit” is ambiguous; I assume you mean working against other goals, like happiness or other people not dying. But since optimizing for one thing means not optimizing for others, every goal has this property relative to every other (for an ideal agent). Still, the concept seems very useful; any thoughts on how to formalize it?
I don’t really have any ideas other than the “negative net sum” worth I mentioned above, but then that just begs the question of what metric one is using to measure worth.
“Not to my benefit” is ambiguous; I assume you mean working against other goals, like happiness or other people not dying. But since optimizing for one thing means not optimizing for others, every goal has this property relative to every other (for an ideal agent). Still, the concept seems very useful; any thoughts on how to formalize it?
I don’t really have any ideas other than the “negative net sum” worth I mentioned above, but then that just begs the question of what metric one is using to measure worth.