Re: Actually this isn’t an example of the AI wireheading (directly adjusting a ‘reward counter’ or positive reinforcer), just a description of a utility function that doesn’t unambiguously pick out what human designers might want.
The idea of the utility function is to create wealth. You seem to be arguing that it would not maximise real weath, but instead the value of a counter representing wealth. If I didn’t express things clearly enough, the idea is to maximise actual access to the real world resources—and not merely the value of some internal register symbolically representing that.
The usual anti-wirehead argument applies to those who would argue that the value of a counter is what will actually be maximised.
You now also seem to be arguing that wealth can’t usefully be measured:
Defining wealth in any sort of precise way that captures what a human is aiming for will involve huge numbers of value-laden decisions, like how to value such items.
So what? It doesn’t have to be particularly precise to be effective. Number of grams of gold you could buy would work pretty well. If you were doing it for real, you’d create a portfolio containing the value of each element, space and energy in proportion to your expected needs—but it would matter relatively little how the elements were weighted. I mean: how much would you really care if you wound up with slightly more gold than you could use—provided you were still a billionaire?
IMHO, the idea that wealth can’t usefully be measured is one which is not sufficiently worthwhile to merit further discussion.
Re: Actually this isn’t an example of the AI wireheading (directly adjusting a ‘reward counter’ or positive reinforcer), just a description of a utility function that doesn’t unambiguously pick out what human designers might want.
The idea of the utility function is to create wealth. You seem to be arguing that it would not maximise real weath, but instead the value of a counter representing wealth. If I didn’t express things clearly enough, the idea is to maximise actual access to the real world resources—and not merely the value of some internal register symbolically representing that.
The usual anti-wirehead argument applies to those who would argue that the value of a counter is what will actually be maximised.
You now also seem to be arguing that wealth can’t usefully be measured:
So what? It doesn’t have to be particularly precise to be effective. Number of grams of gold you could buy would work pretty well. If you were doing it for real, you’d create a portfolio containing the value of each element, space and energy in proportion to your expected needs—but it would matter relatively little how the elements were weighted. I mean: how much would you really care if you wound up with slightly more gold than you could use—provided you were still a billionaire?
IMHO, the idea that wealth can’t usefully be measured is one which is not sufficiently worthwhile to merit further discussion.