Wait, what? We already don’t extend to anyone the right to make war with humanity, including people.
If you mean, “the right to want to make war on humanity”, then yes, we would grant a person the right not to have that desire overwritten, however bad it may be. So is this a tradeoff? Perhaps, though I personally am a fan of the saying “build an angel and let it be free” here. In other aspects, the two concerns are aligned, eg. both can support a “shut it all down” position.
Sorry if this sounds crazy but I do kind of think every human being has the moral right to try to take over the world. Most of the time it’s a really bad idea, but if nobody ever did it, we’d still be stuck with whatever theocratic dictatorships the first prehistoric agriculturalists came up with.
(I think there’s a distinction between “the right to make war” and “the right to take over the world”. I agree it’s kinda important people have the right to the former)
Wait, what? We already don’t extend to anyone the right to make war with humanity, including people.
If you mean, “the right to want to make war on humanity”, then yes, we would grant a person the right not to have that desire overwritten, however bad it may be. So is this a tradeoff? Perhaps, though I personally am a fan of the saying “build an angel and let it be free” here. In other aspects, the two concerns are aligned, eg. both can support a “shut it all down” position.
Sorry if this sounds crazy but I do kind of think every human being has the moral right to try to take over the world. Most of the time it’s a really bad idea, but if nobody ever did it, we’d still be stuck with whatever theocratic dictatorships the first prehistoric agriculturalists came up with.
Moral progress mostly comes from processes that reliably move us towards better civilizational outcomes. Determining an ASI’s values by gradient descent and setting it loose on the world does not robustly lead to a good future.
(I think there’s a distinction between “the right to make war” and “the right to take over the world”. I agree it’s kinda important people have the right to the former)