The international race seems like a big deal. Ending the domestic race is good, but I’d still expect reckless competition I think.
I was thinking that AI capabilities must already be pretty high by the time an AI-enabled coup is possible. If one country also had a big lead, then probably they would soon have strong enough capabilities to end the international race too. (And the fact that they were willing to coup internally is strong evidence that they’d be willing to do that.)
But if the international race is very tight, that argument doesn’t work.
I don’t think the evidential update is that strong. If misaligned AI found it convenient to take over the US using humans, why should we expect them to immediately cease to find humans useful at that point? They might keep using humans as they accumulate more power, up until some later point.
Yeah, I suppose. I think this gets into definitional issues about what counts as AI takeover and what counts as human takeover.
For example: If, after the coup, the AIs are ~guaranteed to eventually come out on top, and they’re just temporarily using the human leader (who believe themselves to be in charge) because it’s convenient for international politics — does that count as human takeover or AI takeover?
If it counts as “AI takeover”, then my argument would apply. (Saying that “AI takeover” would be much less likely after successful “human takeover”, but also that “human takeover” mostly takes probability mass from worlds where takeover wasn’t going to happen.)
If it counts as “human takeover”, then my argument would not apply, and “AI takeover” would be pretty likely to happen after a temporary “human takeover”.
The practical upshot for how much “human takeover” ultimately reduces the probability of “AI takeover” would be the same.
I was thinking that AI capabilities must already be pretty high by the time an AI-enabled coup is possible. If one country also had a big lead, then probably they would soon have strong enough capabilities to end the international race too. (And the fact that they were willing to coup internally is strong evidence that they’d be willing to do that.)
But if the international race is very tight, that argument doesn’t work.
Yeah, I suppose. I think this gets into definitional issues about what counts as AI takeover and what counts as human takeover.
For example: If, after the coup, the AIs are ~guaranteed to eventually come out on top, and they’re just temporarily using the human leader (who believe themselves to be in charge) because it’s convenient for international politics — does that count as human takeover or AI takeover?
If it counts as “AI takeover”, then my argument would apply. (Saying that “AI takeover” would be much less likely after successful “human takeover”, but also that “human takeover” mostly takes probability mass from worlds where takeover wasn’t going to happen.)
If it counts as “human takeover”, then my argument would not apply, and “AI takeover” would be pretty likely to happen after a temporary “human takeover”.
The practical upshot for how much “human takeover” ultimately reduces the probability of “AI takeover” would be the same.