I don’t understand your point about killing them. An AI with my utility function would certainly kill me. There are more efficient arrangements of matter to produce utility.
Keep reading the morality sequence, My comment came while I still had some confusions which are now dissolved.
I don’t know what you count on utility, but I think an AI with your utility function would preserve that which makes you ‘you’ . (it might do anything with your old matter.) At least until it was ready to do something more interesting with ‘you’ .
Pebble sorters value only things that were not pebble sorters, humans value humans, among other things
I don’t understand your point about killing them. An AI with my utility function would certainly kill me. There are more efficient arrangements of matter to produce utility.
Keep reading the morality sequence, My comment came while I still had some confusions which are now dissolved.
I don’t know what you count on utility, but I think an AI with your utility function would preserve that which makes you ‘you’ . (it might do anything with your old matter.) At least until it was ready to do something more interesting with ‘you’ .
Pebble sorters value only things that were not pebble sorters, humans value humans, among other things