I think I’m just not as afraid of covert projects. In the world that has enough political will for Plan A or compute supply dismantling, I think it’s likely that neither the US nor China even wants to attempt a covert project, because they are both genuinely afraid of AI. And I don’t see who else could attempt a covert project with 100k hidden GPUs.
And once the compute chain dismantling happens, it becomes very obvious that the leading powers are extremely serious about not wanting AI. I think it’s over 70% likely that espionage will uncover the existence of a covert project, and then they will very likely get bombed in the world where everyone was so serious that they already blew up their own EUV machines. I expect that the US and China will be aware of this and not attempt a covert project.
(It’s also my understanding that GPUs probably mostly break after about 5 years of use. That will really hurt the covert project if they can’t source new GPUs, no? I don’t remember this being addressed in the supplement, but maybe I missed something.)
Also, even if a covert project manages to build something smarter than anyone else has, that’s not necessarily catastrophic. If they manage to build such a strong superintelligence that it can build invincible nanobot-armies in a basement without any previous industrial build-out, that’s bad. But if they would just want to do the usual strategy of building robots to build better robots to build more compute and drone armies, that will very likely be visible an be noticed, in a world that otherwise doesn’t have advanced compute an robotics, and will lead to bombing.
My understanding is that in this way, Plan A is more vulnerable to covert projects than the compute dismantling plan: in Plan A, if someone secretly builds a smarter AI than anyone else has, they can use it to devise a plan to bring a few percent of the huge compute clusters outside the easily bombable places, and then hook up the secretly built smart AI to the compute and the already existing huge robot economy and drone army. The path to victory seems much harder in the world without a lot of compute and robots.
In the world that has enough political will for Plan A or compute supply dismantling, I think it’s likely that neither the US nor China even wants to attempt a covert project, because they are both genuinely afraid of AI.
The game theory doesn’t work out this way. In two player models, the US and China can always each get a geopolitical power advantage from developing an AI more powerful than socially optimal, and so have an incentive to defect.
If they believe AI is very dangerous, they will just set the treaty capability cap lower, and still defect on it.
Sure, but it’s worth noting that’s a different reason. IMO fear of AI sets the perceived socially optimal outcome, and honor, collusion, transparency, etc determine whether a treaty holds.
I think I’m just not as afraid of covert projects. In the world that has enough political will for Plan A or compute supply dismantling, I think it’s likely that neither the US nor China even wants to attempt a covert project, because they are both genuinely afraid of AI. And I don’t see who else could attempt a covert project with 100k hidden GPUs.
And once the compute chain dismantling happens, it becomes very obvious that the leading powers are extremely serious about not wanting AI. I think it’s over 70% likely that espionage will uncover the existence of a covert project, and then they will very likely get bombed in the world where everyone was so serious that they already blew up their own EUV machines. I expect that the US and China will be aware of this and not attempt a covert project.
(It’s also my understanding that GPUs probably mostly break after about 5 years of use. That will really hurt the covert project if they can’t source new GPUs, no? I don’t remember this being addressed in the supplement, but maybe I missed something.)
Also, even if a covert project manages to build something smarter than anyone else has, that’s not necessarily catastrophic. If they manage to build such a strong superintelligence that it can build invincible nanobot-armies in a basement without any previous industrial build-out, that’s bad. But if they would just want to do the usual strategy of building robots to build better robots to build more compute and drone armies, that will very likely be visible an be noticed, in a world that otherwise doesn’t have advanced compute an robotics, and will lead to bombing.
My understanding is that in this way, Plan A is more vulnerable to covert projects than the compute dismantling plan: in Plan A, if someone secretly builds a smarter AI than anyone else has, they can use it to devise a plan to bring a few percent of the huge compute clusters outside the easily bombable places, and then hook up the secretly built smart AI to the compute and the already existing huge robot economy and drone army. The path to victory seems much harder in the world without a lot of compute and robots.
The game theory doesn’t work out this way. In two player models, the US and China can always each get a geopolitical power advantage from developing an AI more powerful than socially optimal, and so have an incentive to defect.
If they believe AI is very dangerous, they will just set the treaty capability cap lower, and still defect on it.
I think countries sometimes have honor and follow their commitments even if it’s not locally game theoretically optimal.
Sure, but it’s worth noting that’s a different reason. IMO fear of AI sets the perceived socially optimal outcome, and honor, collusion, transparency, etc determine whether a treaty holds.
.