Constant: “Give a person power, and he no longer needs to compromise with others, and so for him the raison d’etre of morality vanishes and he acts as he pleases.”
If you could do so easily and with complete impunity, would you organize fights to death for your pleasure? Would you even want to? Moreover, humans are often tempted to do things they know they shouldn’t, because they also have selfish desires. AIs don’t if you don’t build it into them. If they really do ultimately care about humanity’s well-being, and don’t take any pleasure from making people obey them, they will keep doing so.
Constant: “Give a person power, and he no longer needs to compromise with others, and so for him the raison d’etre of morality vanishes and he acts as he pleases.”
If you could do so easily and with complete impunity, would you organize fights to death for your pleasure? Would you even want to? Moreover, humans are often tempted to do things they know they shouldn’t, because they also have selfish desires. AIs don’t if you don’t build it into them. If they really do ultimately care about humanity’s well-being, and don’t take any pleasure from making people obey them, they will keep doing so.