Many times opinions how to handle uncertainty get baked into the utility functions. That is a standard naive construction is to say “be risk neutral” and value paperclips linearly for their amount. But I could imagine a policy for which more paperclips is always better but from a default position of 100% 2 paperclips it wouldn’t choose a option of 0.1% 1 paperclips, 49.9% 2 paperclips and 50% 3 paperclips. One can construct a “risk averse” function where the new function can simply be optimised. But does it really mean the new function is not a paper clip maximation function?
You’re absolutely right. I was starting to get at this idea from another of the comments, but you’ve laid out where I’ve gone wrong very clearly. Thank you.
Many times opinions how to handle uncertainty get baked into the utility functions. That is a standard naive construction is to say “be risk neutral” and value paperclips linearly for their amount. But I could imagine a policy for which more paperclips is always better but from a default position of 100% 2 paperclips it wouldn’t choose a option of 0.1% 1 paperclips, 49.9% 2 paperclips and 50% 3 paperclips. One can construct a “risk averse” function where the new function can simply be optimised. But does it really mean the new function is not a paper clip maximation function?
You’re absolutely right. I was starting to get at this idea from another of the comments, but you’ve laid out where I’ve gone wrong very clearly. Thank you.