Who cares about humans exactly? I care about utility.
That is nonsense. Utility is usefulness to people. If there are no humans there is no utility. An AI that could become convinced that “humans are not an efficient way to generate utility” would be what is referred to as a paperclipper.
This is why I don’t like the utile jargon. It makes it sound as though utility was something that could be measured independently of human emotions. Perhaps some kind of substance. But if statements about utility are not translated back to statements about human action or goals then they are completely meaningless.
Utility is usefulness to people. If there are no humans there is no utility.
Utility is goodness measured according to some standard of goodness; that standard doesn’t have to reference human beings. In my most optimistic visions of a far future, human values outlive the human race.
Utility is goodness measured according to some standard of goodness; that standard doesn’t have to reference human beings. In my most optimistic visions of a far future, human values outlive the human race.
Are we using the same definition of “human being”? We would not have to be biologically identical to what we are now in order to be people. But human values without humans also sounds meaningless to me. There are no values atoms or goodness atoms sitting around somewhere. To be good or to be valuable something must be good or valuable by the standards of some person. So there would have to be somebody around to do the valuing. But the standards don’t have to be explicit or objective.
Utility as I care about it is probably the result of information processing. Not clear why information should only be able to be processed in that way by human type minds, let alone fleshy ones.
That is nonsense. Utility is usefulness to people. If there are no humans there is no utility. An AI that could become convinced that “humans are not an efficient way to generate utility” would be what is referred to as a paperclipper.
This is why I don’t like the utile jargon. It makes it sound as though utility was something that could be measured independently of human emotions. Perhaps some kind of substance. But if statements about utility are not translated back to statements about human action or goals then they are completely meaningless.
Utility is goodness measured according to some standard of goodness; that standard doesn’t have to reference human beings. In my most optimistic visions of a far future, human values outlive the human race.
Are we using the same definition of “human being”? We would not have to be biologically identical to what we are now in order to be people. But human values without humans also sounds meaningless to me. There are no values atoms or goodness atoms sitting around somewhere. To be good or to be valuable something must be good or valuable by the standards of some person. So there would have to be somebody around to do the valuing. But the standards don’t have to be explicit or objective.
Utility as I care about it is probably the result of information processing. Not clear why information should only be able to be processed in that way by human type minds, let alone fleshy ones.