Like the word “rational” is sometimes used instead of “optimal” or “good”, words “utility function” are probably used to mean “good” or “our values” or something like that.
Therefore, analogically to the suggestion of only using the word ‘rational’ when talking about cognitive algorithms and thinking techniques, we should only use the words ‘utility function’ when talking about computer programs. When speaking about humans, “good / better / the best” probably expresses what we need well enough.
Except … maybe “utility function” need not apply to AI either. Why suppose that the best way to get what we want from an AI is to give it a utility function? Maybe AIs that do have utility functions are generally more dangerous than certain classes of AIs that don’t.
Like the word “rational” is sometimes used instead of “optimal” or “good”, words “utility function” are probably used to mean “good” or “our values” or something like that.
Therefore, analogically to the suggestion of only using the word ‘rational’ when talking about cognitive algorithms and thinking techniques, we should only use the words ‘utility function’ when talking about computer programs. When speaking about humans, “good / better / the best” probably expresses what we need well enough.
Except … maybe “utility function” need not apply to AI either. Why suppose that the best way to get what we want from an AI is to give it a utility function? Maybe AIs that do have utility functions are generally more dangerous than certain classes of AIs that don’t.