Presumably, Eliezer’s simulated planet of Einsteins would be able to infer everything about the tentacle-creatures’ morality simply based on the creatures’ biology and evolutionary past. So I think this hypothetical super-AI could in fact figure out what morality humans subscribe to.
Is anyone else terrified by the notion of a super-AI that looked at human history and figured out what morality humans actually subscribed to, rather than claimed to, and then instantiated it? I can only hope that the AI gives enough weight to the people who lived the sorts of quiet lives that don’t make the history books. Looking to our evolutionary past is also a scary notion.
Presumably, Eliezer’s simulated planet of Einsteins would be able to infer everything about the tentacle-creatures’ morality simply based on the creatures’ biology and evolutionary past. So I think this hypothetical super-AI could in fact figure out what morality humans subscribe to.
Is anyone else terrified by the notion of a super-AI that looked at human history and figured out what morality humans actually subscribed to, rather than claimed to, and then instantiated it? I can only hope that the AI gives enough weight to the people who lived the sorts of quiet lives that don’t make the history books. Looking to our evolutionary past is also a scary notion.