Question: Regardless of the degree to which this is true, if everyone collectively assumed that Valence Utilitarianism (every conscious experience has value (positive or negative, depending on pleasantness/unpleasantness), each action’s utility is the sum of all value it causes / changes / prevents) was universally true, how much would that change about Friendly AI research?
It would surely create new failure modes for Friendliness: e.g. kill all humans to spawn a trillion barely sentients copy of an AI. But on the other hand I would say: not much. You’ll still have the hard problems of instilling a very fragile ontology (what is a conscious being, from the bottom up?), analyzing the consequences of the actions performed by the AI, judging carefully which values it is fulfilling or breaking and how much, etc.
Question: Regardless of the degree to which this is true, if everyone collectively assumed that Valence Utilitarianism (every conscious experience has value (positive or negative, depending on pleasantness/unpleasantness), each action’s utility is the sum of all value it causes / changes / prevents) was universally true, how much would that change about Friendly AI research?
It would surely create new failure modes for Friendliness: e.g. kill all humans to spawn a trillion barely sentients copy of an AI.
But on the other hand I would say: not much. You’ll still have the hard problems of instilling a very fragile ontology (what is a conscious being, from the bottom up?), analyzing the consequences of the actions performed by the AI, judging carefully which values it is fulfilling or breaking and how much, etc.