There are only 2 possible options: Either, in preserving human values, you wish to prevent, forever and all time, the development of any wider range of cognitive experiences and concomitant new values; or your notion of “human values” is so general as to encompass such new developments.
It could encompass some such new developments but not others.
If it can encompass the development of new cognitive experiences, that means that the utility function is not expressed in terms of current cognitive experiences. So what is it expressed in terms of? And what is it preserving?
It could encompass some such new developments but not others.
If it can encompass the development of new cognitive experiences, that means that the utility function is not expressed in terms of current cognitive experiences. So what is it expressed in terms of? And what is it preserving?