It’s like with unfair coins: there are unfair utility functions as well. ;-) You’d have to look at what the utility could be, as with the prior probability of Omega’s coin. All the AI can do is adapt its utility to environment, while there is also a “prior” utility, that describes how it looks at all possible environments, how it chose this particular utility for this particular environment.
It’s like with unfair coins: there are unfair utility functions as well. ;-) You’d have to look at what the utility could be, as with the prior probability of Omega’s coin. All the AI can do is adapt its utility to environment, while there is also a “prior” utility, that describes how it looks at all possible environments, how it chose this particular utility for this particular environment.