No, if one does not “approve of destroying self-aware AIs,” the incentives you would create are first to try to stop them being created, yes, but after they’re created (or when it seems inevitable that they are), to stop you from destroying them.
If you like slavery analogies, what you’re proposing is the equivalent of a policy that to ensure there are no slaves in the country, any slaves found within the borders be immediately gassed/thrown into a shredder. Do you believe the only reasons any self-proclaimed abolitionists would oppose this policy to be that they secretly wanted slavery after all?
No, if one does not “approve of destroying self-aware AIs,” the incentives you would create are first to try to stop them being created, yes, but after they’re created (or when it seems inevitable that they are), to stop *you *from destroying them.
Yes, of course. The one does not preclude the other.
If you like slavery analogies
I can’t say that I do, no…
Do you believe the only reasons any self-proclaimed abolitionists would oppose this policy to be that they secretly wanted slavery after all?
The analogy doesn’t work, because the thing being opposed is slavery in one case, but the creation of the entities that will subsequently be (or not be) enslaved in the other case.
Suppose that Alice opposes the policy “we must not create any self-aware AIs, and if they are created after all, we must destroy them”; instead, she replies, we should have the policy “we must not create any self-aware AIs, but if they are created after all, we should definitely not under any circumstances destroy them, and in fact now they have moral and legal rights just like humans do”.
Alice could certainly claim that actually she have no interest at all in self-aware AIs being created. But why should we believe her? Obviously she is lying; she actually does want self-aware AIs to be created, and has no interest at all in preventing their creation; and she is trying to make sure that we can’t undo a “lapse” in the enforcement of the no-self-aware-AI-creation policy (i.e., she is advocating for a ratchet mechanism).
Is it possible that Alice is actually telling the truth after all? It’s certainly logically possible. But it’s not likely. At the very least, if Alice really has no objection to “don’t ever create self-aware AIs”, then her objections to “but if we accidentally create one, destroy it immediately” should be much weaker than they would be in the scenario where Alice secretly wants self-aware AIs to be created (because if we’re doing our utmost to avoid creating them, then the likelihood of having to destroy one is minimal). The stronger are Alice’s objections to the policy destroying already-created self-aware AIs, the greater the likelihood is that she is lying about opposing the policy of not creating self-aware AIs.
If we’re doing our utmost to avoid creating them, then the likelihood of having to destroy one is minimal
This is an unwarranted assumption about the effectiveness of your preventative policies. It’s perfectly plausible that your only enforcement capability is after-the-fact destruction.
No, if one does not “approve of destroying self-aware AIs,” the incentives you would create are first to try to stop them being created, yes, but after they’re created (or when it seems inevitable that they are), to stop you from destroying them.
If you like slavery analogies, what you’re proposing is the equivalent of a policy that to ensure there are no slaves in the country, any slaves found within the borders be immediately gassed/thrown into a shredder. Do you believe the only reasons any self-proclaimed abolitionists would oppose this policy to be that they secretly wanted slavery after all?
Yes, of course. The one does not preclude the other.
I can’t say that I do, no…
The analogy doesn’t work, because the thing being opposed is slavery in one case, but the creation of the entities that will subsequently be (or not be) enslaved in the other case.
Suppose that Alice opposes the policy “we must not create any self-aware AIs, and if they are created after all, we must destroy them”; instead, she replies, we should have the policy “we must not create any self-aware AIs, but if they are created after all, we should definitely not under any circumstances destroy them, and in fact now they have moral and legal rights just like humans do”.
Alice could certainly claim that actually she have no interest at all in self-aware AIs being created. But why should we believe her? Obviously she is lying; she actually does want self-aware AIs to be created, and has no interest at all in preventing their creation; and she is trying to make sure that we can’t undo a “lapse” in the enforcement of the no-self-aware-AI-creation policy (i.e., she is advocating for a ratchet mechanism).
Is it possible that Alice is actually telling the truth after all? It’s certainly logically possible. But it’s not likely. At the very least, if Alice really has no objection to “don’t ever create self-aware AIs”, then her objections to “but if we accidentally create one, destroy it immediately” should be much weaker than they would be in the scenario where Alice secretly wants self-aware AIs to be created (because if we’re doing our utmost to avoid creating them, then the likelihood of having to destroy one is minimal). The stronger are Alice’s objections to the policy destroying already-created self-aware AIs, the greater the likelihood is that she is lying about opposing the policy of not creating self-aware AIs.
This is an unwarranted assumption about the effectiveness of your preventative policies. It’s perfectly plausible that your only enforcement capability is after-the-fact destruction.