I don’t think the post requires the unsafe ASI to breach containment. The frontier AI being run on the labs‘ own servers could itself be unsafe, successfully scheming without detection. Or it could be corrigible and still be an x/s-risk from humans enacting biorisk, human takeover using the AI, or gradual disempowerment.
Even if a leaked unsafe ASI has extremely constrained compute, it still can enact asymmetric attacks that work within those constraints. How much compute would it need to manipulate someone into—or just help them to—obtain and release smallpox? How much would it need to effectively sabotage safety research or the creation of a safe ASI?
And even if you are actually succeeding at building a safe ASI, it is still very likely to be harder and to take more time to build than an unsafe one. In the time window between when the leading lab could’ve built an unsafe ASI and them building a safe ASI, their competitors may just go ahead and build the unsafe ASI themselves, letting it run on their servers. And then the unsafe ASI has first mover advantage, with plenty of compute to spare, and probably wins.
I don’t think the post requires the unsafe ASI to breach containment. The frontier AI being run on the labs‘ own servers could itself be unsafe, successfully scheming without detection. Or it could be corrigible and still be an x/s-risk from humans enacting biorisk, human takeover using the AI, or gradual disempowerment.
Even if a leaked unsafe ASI has extremely constrained compute, it still can enact asymmetric attacks that work within those constraints. How much compute would it need to manipulate someone into—or just help them to—obtain and release smallpox? How much would it need to effectively sabotage safety research or the creation of a safe ASI?
And even if you are actually succeeding at building a safe ASI, it is still very likely to be harder and to take more time to build than an unsafe one. In the time window between when the leading lab could’ve built an unsafe ASI and them building a safe ASI, their competitors may just go ahead and build the unsafe ASI themselves, letting it run on their servers. And then the unsafe ASI has first mover advantage, with plenty of compute to spare, and probably wins.